CoChat
Sign Up
Cited by 531Information Retrieval and Search BehaviorRecommender Systems and TechniquesMachine Learning and Algorithms

Unbiased Learning-to-Rank with Biased Feedback

Open access
Joachims, Thorsten, Swaminathan, Adith, Schnabel, Tobias·2017·Proceedings of the Tenth ACM International Conference on Web Search and Data Mining
DOI 10.1145/3018661.3018699
Open this paper in CoChat — free accountOpen PDF

Abstract

Implicit feedback (e.g., clicks, dwell times, etc.) is an abundant source of data in human-interactive systems. While implicit feedback has many advantages (e.g., it is inexpensive to collect, user centric, and timely), its inherent biases are a key obstacle to its effective use. For example, position bias in search rankings strongly influences how many clicks a result receives, so that directly using click data as a training signal in Learning-to-Rank (LTR) methods yields sub-optimal results. To overcome this bias problem, we present a counterfactual inference framework that provides the theoretical basis for unbiased LTR via Empirical Risk Minimization despite biased data. Using this framework, we derive a Propensity-Weighted Ranking SVM for discriminative learning from implicit feedback, where click models take the role of the propensity estimator. In contrast to most conventional approaches to de-biasing the data using click models, this allows training of ranking functions even in settings where queries do not repeat. Beyond the theoretical support, we show empirically that the proposed learning method is highly effective in dealing with biases, that it is robust to noise and propensity model misspecification, and that it scales efficiently. We also demonstrate the real-world applicability of our approach on an operational search engine, where it substantially improves retrieval performance.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3018661.3018699

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Attributing a delayed, sparse task outcome to decisions along a trajectory of writing, retention, retrieval, and use is a counterfactual estimation problem.

    “attributing a delayed, sparse task outcome to decisions along a trajectory of writing, retention, retrieval, and use is a counterfactual estimation problem, for which IR's work on counterfactual and online learning to rank [31] provides a relevant toolkit.”

    From Agentic Memory Management Is the Next Frontier of Information Retrieval (2026) · cites this paper as [31]

    The Introduction and Section 4 describe a counterfactual learning-to-rank toolkit for estimating performance from biased, partially observed feedback. But the Introduction and Related Work distinguish the paper’s method from online learning algorithms; it does not offer an online learning-to-rank method. The writing-to-use trajectory is the citing authors’ application of the toolkit, not a setting studied by this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Extracting signal from noisy implicit evidence is the problem that click models and unbiased learning to rank were built for.

    “Extracting signal from noisy implicit evidence is the problem that click models and unbiased learning to rank were built for [17, 31].”

    From Agentic Memory Management Is the Next Frontier of Information Retrieval (2026) · cites this paper as [31]

  • IR has shown how a proxy that diverges from the objective can distort a system and how implicit feedback can mislead when position and selection biases are ignored.

    “IR has shown how a proxy that diverges from the objective can distort a system and how implicit feedback can mislead when position and selection biases are ignored [31].”

    From Agentic Memory Management Is the Next Frontier of Information Retrieval (2026) · cites this paper as [31]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 28

  1. A. Borisov, I. Markov, M. de Rijke, and P. Serdyukov. A neural click model for web search. In Proceedings of the 25th International Conference on World Wide Web, pages 531-541, 2016.
  2. O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large-scale validation and analysis of interleaved search evaluation. ACM Transactions on Information Systems (TOIS), 30(1):6:1-6:41, 2012.
  3. O. Chapelle and Y. Zhang. A dynamic bayesian network click model for web search ranking. In International Conference on World Wide Web (WWW), pages 1-10. ACM, 2009.
  4. N. Craswell, O. Zoeter, M. Taylor, and B. Ramsey. An experimental comparison of click position-bias models. In International Conference on Web Search and Data Mining (WSDM), pages 87-94. ACM, 2008.
  5. K. Hofmann, A. Schuth, S. Whiteson, and M. de Rijke. Reusing historical interaction data for faster online learning to rank for ir. In International Conference on Web Search and Data Mining (WSDM), pages 183-192, 2013.
  6. D. G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47(260):663-685, 1952.
  7. T. Joachims. Optimizing search engines using clickthrough data. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 133-142, 2002.
  8. T. Joachims. Training linear SVMs in linear time. In ACM SIGKDD International Conference On Knowledge Discovery and Data Mining (KDD), pages 217-226, 2006.
  9. T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Transactions on Information Systems (TOIS), 25(2), April 2007.
  10. J. Langford, A. Strehl, and J. Wortman. Exploration scavenging. In Proceedings of the 25th International Conference on Machine Learning, pages 528-535, 2008.
  11. L. Li, W. Chu, J. Langford, and X. Wang. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In International Conference on Web Search and Data Mining (WSDM), pages 297-306, 2011.
  12. T.-Y. Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225-331, Mar. 2009.
  13. K. Raman and T. Joachims. Learning socially optimal information systems from egoistic users. In European Conference on Machine Learning (ECML), pages 128-144, 2013.
  14. K. Raman, T. Joachims, P. Shivaswamy, and T. Schnabel. Stable coactive learning via perturbation. In International Conference on Machine Learning (ICML), pages 837-845, 2013.
  15. M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: Estimating the click-through rate for new ads. In International Conference on World Wide Web (WWW), pages 521-530. ACM, 2007.
  16. P. R. Rosenbaum and D. B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.
  17. T. Schnabel, A. Swaminathan, P. Frazier, and T. Joachims. Unbiased comparative evaluation of ranking functions. In ACM International Conference on the Theory of Information Retrieval (ICTIR), 2016.
  18. T. Schnabel, A. Swaminathan, A. Singh, N. Chandak, and T. Joachims. Recommendations as treatments: Debiasing learning and evaluation. In International Conference on Machine Learning (ICML), 2016.
  19. A. Schuth, H. Oosterhuis, S. Whiteson, and M. de Rijke. Multileave gradient descent for fast online learning to rank. In International Conference on Web Search and Data Mining (WSDM), pages 457-466, 2016.
  20. K. Sparck-Jones and C. J. V. Rijsbergen. Report on the need for and provision of an "ideal" information retrieval test collection. Technical report, University of Cambridge, 1975.
  21. A. L. Strehl, J. Langford, L. Li, and S. Kakade. Learning from logged implicit exploration data. In Conference on Neural Information Processing Systems (NIPS), pages 2217-2225, 2010.
  22. A. Swaminathan and T. Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization. Journal of Machine Learning Research (JMLR), 16:1731-1755, Sep 2015.
  23. V. Vapnik. Statistical Learning Theory. Wiley, Chichester, GB, 1998.
  24. L. Wang, J. J. Lin, and D. Metzler. A cascade ranking model for efficient ranked retrieval. In ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 105-114, 2011.
  25. X. Wang, M. Bendersky, D. Metzler, and M. Najork. Learning to rank with selection bias in personal search. In ACM Conference on Research and Development in Information Retrieval (SIGIR). ACM, 2016.
  26. Y. Wang, D. Yin, L. Jie, P. Wang, M. Yamada, Y. Chang, and Q. Mei. Beyond ranking: Optimizing whole-page presentation. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, WSDM '16, pages 103-112, 2016.
  27. Y. Yue and T. Joachims. Interactively optimizing information retrieval systems as a dueling bandits problem. In International Conference on Machine Learning (ICML), pages 151-159, 2009.
  28. Y. Yue, R. Patel, and H. Roehrig. Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data. In International Conference on World Wide Web (WWW), pages 1011-1018. ACM, 2010.

Cited by

Sorted by citations

Authors
Joachims, T., Swaminathan, A., & Schnabel, T.
Affiliations
Cornell University (US)
ORCID
Venue
Proceedings of the Tenth ACM International Conference on Web Search and Data Mining
Publisher
ACM
Published
2017-02-02
Pages
781-789
DOI
10.1145/3018661.3018699
OpenAlex
W2507134384
Cited by
531
References
2
Citations by year
2026: 46 · 2025: 47 · 2024: 79 · 2023: 64 · 2022: 70 · 2021: 76 · 2020: 66 · 2019: 40 · 2018: 35 · 2017: 8
FWCI
30.85
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor, 365 d)
Locations
ACM International Conference on Web Search and Data Mining (WSDM) (publishedVersion)
Funders
National Science Foundation (IIS-1247637, IIS-1513692, IIS-1615706)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
learning to rank, implicit feedback, position bias, bias mitigation, counterfactual inference, empirical risk minimization, click models, ranking SVM, search engine retrieval, user interaction data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review