The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
SoftRank: optimizing non-smooth rank metrics
Abstract
We address the problem of learning large complex ranking functions. Most IR applications use evaluation metrics that depend only upon the ranks of documents. However, most ranking functions generate document scores, which are sorted to produce a ranking. Hence IR metrics are innately non-smooth with respect to the scores, due to the sort. Unfortunately, many machine learning algorithms require the gradient of a training objective in order to perform the optimization of the model parameters,and because IR metrics are non-smooth,we need to find a smooth proxy objective that can be used for training. We present a new family of training objectives that are derived from the rank distributions of documents, induced by smoothed scores. We call this approach SoftRank. We focus on a smoothed approximation to Normalized Discounted Cumulative Gain (NDCG), called SoftNDCG and we compare it with three other training objectives in the recent literature. We present two main results. First, SoftRank yields a very good way of optimizing NDCG. Second, we show that it is possible to achieve state of the art test set NDCG results by optimizing a soft NDCG objective on the training set with a different discount function
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
Learning to rank: from pairwise approach to listwise approach
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273513 · Cited by 1,960
Cited by
Learning to rank: from pairwise approach to listwise approach
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273513 · Cited by 1,960
Learning diverse rankings with multi-armed bandits
Radlinski, F., Kleinberg, R., & Joachims, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390255 · Cited by 503
End-to-End Training of Object Class Detectors for Mean Average Precision
Henderson, P., & Ferrari, V. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54193-8_13 · Cited by 373
The LambdaLoss Framework for Ranking Metric Optimization
Wang, X., Li, C., Golbandi, N., Bendersky, M., & Najork, M. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271784 · Cited by 136
A General Framework for Counterfactual Learning-to-Rank
Agarwal, A., Takatsu, K., Zaitsev, I., & Joachims, T. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331202 · Cited by 115
Metadata
- Authors
- Taylor, M., Guiver, J., Robertson, S., & Minka, T.
- Venue
- Proceedings of the international conference on Web search and web data mining - WSDM '08
- Publisher
- ACM Press
- Published
- 2008
- Pages
- 77
- OpenAlex
- W2059001985
- Cited by
- 322
- References
- 2
- Type
- conference abstract
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SoftRank: optimizing non-smooth rank metrics. https://app.cochat.ai/papers/10.1145/1341531.1341544
- The paper
- Taylor, M., Guiver, J., Robertson, S., & Minka, T. (2008). SoftRank: optimizing non-smooth rank metrics. Proceedings of the international conference on Web search and web data mining - WSDM '08. https://doi.org/10.1145/1341531.1341544
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.