The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback
Open accessAbstract
Recommender systems widely use implicit feedback such as click data because of its general availability. Although the presence of clicks signals the users' preference to some extent, the lack of such clicks does not necessarily indicate a negative response from the users, as it is possible that the users were not exposed to the items (positive-unlabeled problem). This leads to a difficulty in predicting the users' preferences from implicit feedback. Previous studies addressed the positive-unlabeled problem by uniformly upweighting the loss for the positive feedback data or estimating the confidence of each data having relevance information via the EM-algorithm. However, these methods failed to address the missing-not-at-random problem in which popular or frequently recommended items are more likely to be clicked than other items even if a user does not have a considerable interest in them. To overcome these limitations, we first define an ideal loss function to be optimized to realize recommendations that maximize the relevance and propose an unbiased estimator for the ideal loss. Subsequently, we analyze the variance of the proposed unbiased estimator and further propose a clipped estimator that includes the unbiased estimator as a special case. We demonstrate that the clipped estimator is expected to improve the performance of the recommender system, by considering the bias-variance trade-off. We conduct semi-synthetic and real-world experiments and demonstrate that the proposed method largely outperforms the baselines. In particular, the proposed method works better for less popular items that are less frequently observed in the training data. The findings indicate that the proposed method can better achieve the objective of recommending items with the highest relevance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,673
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Larian Studios Dev Team (2024) · doi:10.1145/3292500.3330701 · Cited by 8,618
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Collaborative Filtering for Implicit Feedback Datasets
Hu, Y., Koren, Y., & Volinsky, C. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.22 · Cited by 3,170
Learning classifiers from only positive and unlabeled data
Elkan, C., & Noto, K. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401920 · Cited by 1,028
Conference on Neural Information Processing Systems
Liu, L., Zhou, T., Long, G., Jiang, J., & Zhang, C. (2019) · UTS ePRESS (University of Technology Sydney) · Cited by 593
Unbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018699 · Cited by 531 · cited 2×
“As in the information retrieval field [11, 12, 17, 27], one should estimate the relevance of an item to a user from implicit feedback to improve user experience, because one can recommend items that users are genuinely interested in by accurately predicting its relevance.”
BackgroundTop-K Off-Policy Correction for a REINFORCE Recommender System
Chen, M., Beutel, A., Covington, P., Jain, S., Belletti, F., & Chi, E. H. (2019) · Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining · doi:10.1145/3289600.3290999 · Cited by 393
Modeling User Exposure in Recommendation
Liang, D., Charlin, L., McInerney, J., & Blei, D. M. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883090 · Cited by 362
Offline A/B Testing for Recommender Systems
Gilotte, A., Calauzènes, C., Nédelec, T., Abraham, A., & Dollé, S. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159687 · Cited by 306
Position Bias Estimation for Unbiased Learning to Rank in Personal Search
Wang, X., Golbandi, N., Bendersky, M., Metzler, D., & Najork, M. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159732 · Cited by 268
Factorization Meets the Item Embedding: Regularizing Matrix Factorization with Item Co-occurrence
Liang, D., Altosaar, J., Charlin, L., & Blei, D. M. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959182 · Cited by 241
Causal embeddings for recommendation
Bonner, S., & Vasile, F. (2018) · Proceedings of the 12th ACM Conference on Recommender Systems · doi:10.1145/3240323.3240360 · Cited by 234
The self-normalized estimator for counterfactual learning
Swaminathan, A., & Joachims, T. (2015) · Cited by 209
Unbiased offline recommender evaluation for missing-not-at-random implicit feedback
Yang, L., Cui, Y., Xuan, Y., Wang, C., Belongie, S., & Estrin, D. (2018) · Proceedings of the 12th ACM Conference on Recommender Systems · doi:10.1145/3240323.3240355 · Cited by 207
Shepherd, B. E., Jarrett, R. T., & Fu, L. (2016) · Biometrics · doi:10.1111/biom.12615 · Cited by 161
Learning from Positive and Unlabeled Examples with Different Data Distributions
Li, X., & Liu, B. (2005) · Lecture notes in computer science · doi:10.1007/11564096_24 · Cited by 111
Recommending Based on Implicit Feedback
Jannach, D., Lerche, L., & Zanker, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-90092-6_14 · Cited by 106
Counterfactual Evaluation and Learning for Search, Recommendation and Ad Placement
Joachims, T., & Swaminathan, A. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2914803 · Cited by 53
Doubly Robust Prediction and Evaluation Methods Improve Uplift Modeling for Observational Data
Saito, Y., Sakata, H., & Nakata, K. (2019) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611975673.53 · Cited by 19
Self-citationSpiral of Silence in Recommender Systems
Liu, D., Lin, C., Zhang, Z., Xiao, Y., & Tong, H. (2019) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3289600.3291003 · Cited by 18
Cited by
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
Popularity Bias in Dynamic Recommendation
Zhu, Z., He, Y., Zhao, X., & Caverlee, J. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467376 · Cited by 82
Huang, J., Oosterhuis, H., Mansoury, M., van Hoof, H., & de Rijke, M. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657749 · Cited by 17
Pan, Q., Li, S., Zheng, T., Ma, F., & Huang, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123227 · Cited by 7
Metadata
- Authors
- Saito, Y., Yaginuma, S., Nishino, Y., Sakata, H., & Nakata, K.
- Venue
- Proceedings of the 13th International Conference on Web Search and Data Mining
- Publisher
- ACM
- Published
- 2020-01-20
- Pages
- 501-509
- OpenAlex
- W2998534896
- Cited by
- 295
- References
- 23
- FWCI
- 27.98
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.