The DOI matches its Crossref record; 90 of the 90 references it cites were found in a registry or an index.
Learning to rank: from pairwise approach to listwise approach
Abstract
The paper is concerned with learning to rank, which is to construct a model or a function for ranking objects. Learning to rank is useful for document retrieval, collaborative filtering, and many other applications. Several methods for learning to rank have been proposed, which take object pairs as 'instances' in learning. We refer to them as the pairwise approach in this paper. Although the pairwise approach offers advantages, it ignores the fact that ranking is a prediction task on list of objects. The paper postulates that learning to rank should adopt the listwise approach in which lists of objects are used as 'instances' in learning. The paper proposes a new probabilistic method for the approach. Specifically it introduces two probability models, respectively referred to as permutation probability and top k probability, to define a listwise loss function for learning. Neural Network and Gradient Descent are then employed as model and algorithm in the learning method. Experimental results on information retrieval show that the proposed listwise approach performs better than the pairwise approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Baeza-Yates, R. A., & Ribeiro-Neto, B. A. (1999) · Cited by 11,505
Optimizing search engines using clickthrough data
Joachims, T. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775067 · Cited by 3,873 · cited 6×
“Second, the training instances of document pairs can be easily obtained in certain scenarios (Joachims, 2002).”
BackgroundMethodInfluentialLearning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
Pfanzagl, J., & Luce, R. D. (1962) · Econometrica · doi:10.2307/1911299 · Cited by 2,469 · cited 1×
“For example, following the work by Luce (1959), Plackett (1975) de.ned probability models on ranking lists of objects.”
MethodLearning to rank for information retrieval
Liu, T.-Y. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835676 · Cited by 1,856
Self-citationMaking Large-Scale Support Vector Machine Learning Practical
Joachims, T. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0015 · Cited by 1,613
IR evaluation methods for retrieving highly relevant documents
Järvelin, K., & Kekäläinen, J. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130374 · Cited by 1,444
Large Margin Rank Boundaries for Ordinal Regression
Herbrich, R., Graepel, T., & Obermayer, K. (2000) · Advances in Large-Margin Classifiers · doi:10.7551/mitpress/1113.003.0010 · Cited by 1,074
IR evaluation methods for retrieving highly relevant documents
Järvelin, K., & Kekäläinen, J. (2000) · Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/345508.345545 · Cited by 967 · cited 2×
“In ranking performance evaluation, we adopted two common IR evaluation measures: Normalized Discounted Cumulative Gain (NDCG) (Jarvelin & Kekanainen, 2000) and Mean Average Precision (MAP)(Baeza-Yates & Ribeiro-Neto, 1999).”
MethodAdaRank: a boosting algorithm for information retrieval
Xu, J., & Li, H. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277809 · Cited by 775
Self-citation
Research works printed but not linked: 5
- Crammer K., 2001, Proceedings of NIPS
- Freund Y., 1998, Proceedings of ICML 1998 (pp. 170--178)
- 10.5555/188490.188557 · doi:10.5555/188490.188557
- Lebanon G., 2002, Proceedings of ICML 2002 (pp. 363--370)
- Shashua A., 2002, Proceedings of NIPS
Cited by
Learning to rank for information retrieval
Liu, T.-Y. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835676 · Cited by 1,856
Self-citationLearning to Rank for Information Retrieval
Liu, T.-Y. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000016 · Cited by 1,568
Self-citationParikh, D., & Grauman, K. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126281 · Cited by 851
“While alternative list-based learning to rank techniques are available [23], we choose the pairwise learning technique as described in Section 3.”
MethodReciprocal rank fusion outperforms condorcet and individual rank learning methods
Cormack, G. V., Clarke, C. L. A., & Buettcher, S. (2009) · Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1571941.1572114 · Cited by 797
“Condorcet 0.5917 (0.56 - 0.62) 0.0134 (0.00 - 0.02) .004 CombMNZ 0.6107 (0.58 - 0.64) -0.0056 (-0.01 - 0) .2 ListNet [ 1 ] 0.5846 (0.56 - 0.61) 0.0205 (0.01 - 0.03) .001”
InfluentialEntity Linking with a Knowledge Base: Issues, Techniques, and Solutions
Shen, W., Wang, J., & Han, J. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2327028 · Cited by 761
“entities: the pairwise framework Ranking Perceptron [117] and the listwise framework ListNet [118].”
Method
Metadata
- Authors
- Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H.
- Affiliations
- Tsinghua University (CN)
- Microsoft Research Asia (China) (CN)
- National Taiwan University (TW)
- Venue
- Proceedings of the 24th international conference on Machine learning
- Publisher
- ACM
- Published
- 2007-06-20
- Pages
- 129-136
- OpenAlex
- W2108862644
- Cited by
- 1,960
- References
- 90
- Citations by year
- 2026: 79 · 2025: 98 · 2024: 115 · 2023: 110 · 2022: 104 · 2021: 151 · 2020: 143 · 2019: 134 · 2018: 141 · 2017: 140 · 2016: 105 · 2015: 115 · 2014: 108 · 2013: 84 · 2012: 62
- FWCI
- 40.69
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- learning to rank, pairwise learning, information retrieval, gradient descent
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning to rank: from pairwise approach to listwise approach. https://app.cochat.ai/papers/10.1145/1273496.1273513
- The paper
- Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007). Learning to rank: from pairwise approach to listwise approach. Proceedings of the 24th international conference on Machine learning. https://doi.org/10.1145/1273496.1273513
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.