The Probabilistic Relevance Framework: BM25 and Beyond
The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
The Probabilistic Relevance Framework (PRF) is a formal framework for document retrieval, grounded in work done in the 1970–1980s, which led to the development of one of the most successful text-retrieval algo¬rithms, BM25. In recent years, research in the PRF has yielded new retrieval models capable of taking into account document meta-data (especially structure and link-graph information). Again, this has led to one of the most successful Web-search and corporate-search algo¬rithms, BM25F. This work presents the PRF from a conceptual point of view, describing the probabilistic modelling assumptions behind the framework and the different ranking algorithms that result from its application: the binary independence model, relevance feedback mod¬els, BM25 and BM25F. It also discusses the relation between the PRF and other statistical models for IR, and covers some related topics, such as the use of non-textual features, and parameter optimisation for models with free parameters.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Most flat baselines also lose ground as pressure rises; BM25 drops from 0 . 808 at 3 -hop to 0 . 425 at 5 / 6 -hop, confirming that the difficulty here is driven by the memory dimension rather than by task length per se.”
From MemGym: a Long-Horizon Memory Environment for LLM Agents (2026) · cites this paper as [38]
In the citing paper
“The combination of vector and BM25 [Robertson and Zaragoza, 2009] retrieval through RRF fusion [Cormack et al., 2009] consistently outperforms any single approach, consistent with findings in hybrid search literature [Ma et al., 2021].”
From Cognis: Context-Aware Memory for Conversational AI Agents (2026) · cites this paper as Robertson and Zaragoza, 2009
In the citing paper
“A streamlined dual-store architecture combining OpenSearch (for documents and native BM25 [Robertson and Zaragoza, 2009] search) with a vector database (for Matryoshka [Kusupati et al., 2022] embeddings at 768D and 256D)”
From Cognis: Context-Aware Memory for Conversational AI Agents (2026) · cites this paper as Robertson and Zaragoza, 2009
In the citing paper
“The BM25 retriever clearly outperformed all dense retrievers.”
From In-Context Retrieval-Augmented Language Models (2023) · cites this paper as Robertson and Zaragoza, 2009
In the citing paper
“a 345M parameter GPT-2 enhanced by In-Context RALM outperforms a 762M parameter GPT-2 when employing an off-the-shelf BM25 retriever (Robertson and Zaragoza, 2009)”
From In-Context Retrieval-Augmented Language Models (2023) · cites this paper as Robertson and Zaragoza, 2009
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Probabilistic latent semantic indexing
Hofmann, T. (1999) · Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/312624.312649 · Cited by 3,976
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
THE PROBABILITY RANKING PRINCIPLE IN IR
ROBERTSON, S. E. (1977) · Journal of Documentation · doi:10.1108/eb026647 · Cited by 1,103
Self-citationRelevance based language models
Lavrenko, V., & Croft, W. B. (2001) · Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/383952.383972 · Cited by 1,037
A probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00015-7 · Cited by 942
Self-citationOn Relevance, Probabilistic Indexing and Information Retrieval
Maron, M. E., & Kuhns, J. L. (1960) · Journal of the ACM · doi:10.1145/321033.321035 · Cited by 907
Probabilistic models of information retrieval based on measuring the divergence from randomness
Amati, G., & Van Rijsbergen, C. J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582416 · Cited by 883
A Markov random field model for term dependencies
Metzler, D., & Croft, W. B. (2005) · Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1076034.1076115 · Cited by 843
Document language models, query models, and risk minimization for information retrieval
Lafferty, J., & Zhai, C. (2001) · Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/383952.383970 · Cited by 530
USING PROBABILISTIC MODELS OF DOCUMENT RETRIEVAL WITHOUT RELEVANCE INFORMATION
CROFT, W. B., & HARPER, D. J. (1979) · Journal of Documentation · doi:10.1108/eb026683 · Cited by 433
The eighth text REtrieval conference (TREC-8)
(2000) · doi:10.6028/nist.sp.500-246 · Cited by 336
Probabilistic Models in Information Retrieval
Fuhr, N. (1992) · The Computer Journal · doi:10.1093/comjnl/35.3.243 · Cited by 315
ON TERM SELECTION FOR QUERY EXPANSION
ROBERTSON, S. E. (1990) · Journal of Documentation · doi:10.1108/eb026866 · Cited by 312
Self-citationA probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00016-9 · Cited by 277
Self-citationInformation retrieval using a singular value decomposition model of latent semantic structure
Furnas, G. W., Deerwester, S., Dumais, S. T., Landauer, T. K., Harshman, R. A., Streeter, L. A., & Lochbaum, K. E. (1988) · Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR '88 · doi:10.1145/62437.62487 · Cited by 257
“Is this document relevant?…probably”: a survey of probabilistic models in information retrieval
Crestani, F., Lalmas, M., Van Rijsbergen, C. J., & Campbell, I. (1998) · ACM Computing Surveys · doi:10.1145/299917.299920 · Cited by 202
Harter, S. P. (1975) · Journal of the American Society for Information Science · doi:10.1002/asi.4630260402 · Cited by 195
Relevance weighting for query independent evidence
Craswell, N., Robertson, S., Zaragoza, H., & Taylor, M. (2005) · Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1076034.1076106 · Cited by 159
Self-citationParsimonious language models for information retrieval
Hiemstra, D., Robertson, S. E., & Zaragoza, H. (2004) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1008992.1009025 · Cited by 128
Self-citationOptimisation methods for ranking functions with multiple parameters
Taylor, M. J. H., Zaragoza, H., Craswell, N., Robertson, S. E., & Burges, C. J. C. (2006) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1183614.1183698 · Cited by 122
Self-citationSome inconsistencies and misidentified modeling assumptions in probabilistic information retrieval
Cooper, W. S. (1995) · ACM Transactions on Information Systems · doi:10.1145/195705.195735 · Cited by 75
Learning to rank for information retrieval (LR4IR 2007)
Joachims, T., Li, H., Liu, T., & Zhai, C. (2007) · ACM SIGIR Forum · doi:10.1145/1328964.1328974 · Cited by 71
Automatic feature selection in the markov random field model for information retrieval
Metzler, D. (2007) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1321440.1321478 · Cited by 54
On rank-based effectiveness measures and optimization
Robertson, S. E., & Zaragoza, H. (2007) · Information Retrieval · doi:10.1007/s10791-007-9025-9 · Cited by 43
Self-citationThreshold Setting and Performance Optimization in Adaptive Filtering
Robertson, S. E. (2002) · Information Retrieval · doi:10.1023/a:1015702129514 · Cited by 27
Self-citationA new unified probabilistic model
Bodoff, D., & Robertson, S. E. (2004) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10398 · Cited by 16
Self-citationLearning to rank for information retrieval (LR4IR 2008)
Li, H., Liu, T., & Zhai, C. (2008) · ACM SIGIR Forum · doi:10.1145/1480506.1480519 · Cited by 10
Research works printed but not linked: 21
- Agarwal, 2005, Proceedings of the NIPS 2005 Workshop on Learning to Rank
- Beaulieu, 1997, Okapi at TREC-5, 143
- Berghen, 2004, CONDOR: A constrained, non-linear, derivative-free parallel optimizer for continuous, high computing load, noisy objective functions
- Blei, 2003, Latent dirichlet allocation, Journal of Machine Learning Research, 3, 993
- Boldi, 2005, MG4J at TREC 2005, 10.6028/NIST.SP.500-266.terabyte-umilano.vigna · doi:10.6028/nist.sp.500-266.terabyte-umilano.vigna
- Craswell, 2005, Microsoft Cambridge at TREC-14: Enterprise track, 10.6028/NIST.SP.500-266.enterprise-microsoft.robertson · doi:10.6028/nist.sp.500-266.enterprise-microsoft.robertson
- Lafferty, 2003, Language Modelling for Information Retrieval, 1
- Metzler, 2009, Information Retrieval in Practice
- Pérez-Agüera, 2008, UCM-Y!R at CLEF 2008 Robust and WSD tasks
- Pérez-Agüera, 2008, Exploiting morphological query structure using genetic optimization
- Robertson, 1983, The unified probabilistic model for IR, 108
- Robertson, 1977, Relevance weighting of search terms, Journal of the American Society for Information Science
- Robertson, 1981, Probabilistic models of indexing and searching, 35
- Robertson, 1994, Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval, 232
- Robertson, 2007, On relevance weights with little relevance information, 16
- Robertson, 1992, Okapi at TREC, 21
- Robertson, 2004, Simple BM25 extension to multiple weighted fields, 42
- Song, 2008, Viewing term proximity from a different perspective, 346
- Tao, 2007, An exploration of proximity measures in information retrieval, 295
- van Os, 2006, PF/Tijah: Text search in an XML database system, 12
- Zaragoza, 2005, Microsoft Cambridge at TREC 2004: Web and HARD track
Cited by
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050
“At inference time, all documents containing words from a query are selected and ranked based on various manually defined formulas 42 .”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827
Metadata
- Authors
- Robertson, S., & Zaragoza, H.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Yahoo (Spain) (ES)
- ORCID
- Robertson, Stephen
- Venue
- Foundations and Trends® in Information Retrieval
- Publisher
- Emerald
- Published
- 2009-09-17
- Volume
- 4
- Issue
- 1-2
- Pages
- 1-174
- ISSN
- 1554-0669 (ISSN-L)
- 1554-0677
- OpenAlex
- W4252076394
- Cited by
- 3,374
- References
- 27
- Citations by year
- 2026: 737 · 2025: 605 · 2024: 389 · 2023: 432 · 2022: 282 · 2021: 229 · 2020: 152 · 2019: 120 · 2018: 118 · 2017: 74 · 2016: 79 · 2015: 50 · 2014: 29 · 2013: 36 · 2012: 22
- FWCI
- 36.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Foundations and Trends® in Information Retrieval (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Mathematics › Theoretical Computer Science › Algorithms and Data Compression
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- BM25, probabilistic retrieval, document structure, link graphs, parameter optimization, web search
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.