The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Recommending what video to watch next: a multitask ranking system
Open accessAbstract
In this paper, we introduce a large scale multi-objective ranking system for recommending what video to watch next on an industrial video sharing platform. The system faces many real-world challenges, including the presence of multiple competing ranking objectives, as well as implicit selection biases in user feedback. To tackle these challenges, we explored a variety of soft-parameter sharing techniques such as Multi-gate Mixture-of-Experts so as to efficiently optimize for multiple ranking objectives. Additionally, we mitigated the selection biases by adopting a Wide & Deep framework. We demonstrated that our proposed techniques can lead to substantial improvements on recommendation quality on one of the world's largest video sharing platforms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
LightGBM: A Highly Efficient Gradient Boosting Decision Tree
(2017) · HAL · Cited by 10,512
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Adaptive Mixtures of Local Experts
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.79 · Cited by 5,384
Adversarial Discriminative Domain Adaptation
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.316 · Cited by 5,154 · cited 1×
“• Adversarial learning: Inspired by the broad adoption of adversarial learning in domain adaptation [37] and machine learning fairness [5], we use a similar technique to introduce an auxiliary task which predicts position shown in training data.”
MethodDeep Neural Networks for YouTube Recommendations
Covington, P., Adams, J., & Sargin, E. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959190 · Cited by 3,539
Wide & Deep Learning for Recommender Systems
Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., et al. (2016) · Proceedings of the 1st Workshop on Deep Learning for Recommender Systems · doi:10.1145/2988450.2988454 · Cited by 3,535
Self-citationDeep Learning Based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2020) · ACM Computing Surveys · doi:10.1145/3285029 · Cited by 3,485
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836 · cited 2×
“For problem formulation, we adopt the learning-to-rank framework [6].”
BackgroundMethodDeep Learning using Rectified Linear Units (ReLU)
Agarap, A. F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08375 · Cited by 2,399
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220007 · Cited by 1,233
Self-citationThe YouTube video recommendation system
Davidson, J., Liebald, B., Liu, J., Nandy, P., Van Vleet, T., Gargi, U., Gupta, S., He, Y., Lambert, M., Livingston, B., et al. (2010) · Proceedings of the fourth ACM conference on Recommender systems · doi:10.1145/1864708.1864770 · Cited by 1,191
Deep & Cross Network for Ad Click Predictions
Wang, R., Fu, B., Fu, G., & Wang, M. (2017) · Proceedings of the ADKDD'17 · doi:10.1145/3124749.3124754 · Cited by 1,161
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · cited 2×
“In MMoE, the softmax gating networks have been reported [32] to have imbalanced expert distribution problem, where gating networks converge to have most zero-utilization on experts.”
BackgroundPractical Lessons from Predicting Clicks on Ads at Facebook
He, X., Pan, J., Jin, O., Xu, T., Liu, B., Xu, T., Shi, Y., Atallah, A., Herbrich, R., Bowers, S., et al. (2014) · Proceedings of the Eighth International Workshop on Data Mining for Online Advertising · doi:10.1145/2648584.2648589 · Cited by 869
A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems
Elkahky, A. M., Song, Y., & He, X. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741667 · Cited by 704
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinski, F., & Gay, G. (2007) · ACM Transactions on Information Systems · doi:10.1145/1229179.1229181 · Cited by 677
Deep Learning based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07435 · Cited by 635
A dynamic bayesian network click model for web search ranking
Chapelle, O., & Zhang, Y. (2009) · Proceedings of the 18th international conference on World wide web · doi:10.1145/1526709.1526711 · Cited by 537
Unbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018699 · Cited by 531 · cited 4×
“Other approaches try to learn a bias term from position and apply it as a normalizer or regularizer [23, 34, 41].”
BackgroundInfluentialWTF: the who to follow service at Twitter
Gupta, P., Goel, A., Lin, J., Sharma, A., Wang, D., & Zadeh, R. (2013) · Proceedings of the 22nd international conference on World Wide Web · doi:10.1145/2488388.2488433 · Cited by 461
TFX: A TensorFlow-Based Production-Scale Machine Learning Platform
Baylor, D., Breck, E., Cheng, H.-T., Fiedel, N., Foo, C. Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., et al. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098021 · Cited by 350
Learning to Rank with Selection Bias in Personal Search
Wang, X., Bendersky, M., Metzler, D., & Najork, M. (2016) · Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval · doi:10.1145/2911451.2911537 · Cited by 284
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
Beutel, A., Chen, J., Zhao, Z., & Chi, E. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00075 · Cited by 274
Self-citationExplainable Recommendation via Multi-Task Learning in Opinionated Text Data
Wang, N., Wang, H., Jia, Y., & Yin, Y. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210010 · Cited by 208
Improving User Topic Interest Profiles by Behavior Factorization
Zhao, Z., Cheng, Z., Hong, L., & Ed, H. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741656 · Cited by 194
Self-citationBatch learning from logged bandit feedback through counterfactual risk minimization
Swaminathan, A., & Joachims, T. (2015) · Journal of Machine Learning Research · Cited by 193
Offline and online evaluation of news recommender systems at swissinfo.ch
Garcin, F., Faltings, B., Donatsch, O., Alazzawi, A., Bruttin, C., & Huber, A. (2014) · Proceedings of the 8th ACM Conference on Recommender systems · doi:10.1145/2645710.2645745 · Cited by 177
Learning Multiple Tasks with Multilinear Relationship Networks
Long, M., Cao, Z., Wang, J., & Yu, P. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02117 · Cited by 173
Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time
Eksombatchai, C., Jindal, P., Liu, J. Z., Liu, Y., Sharma, R., Sugnet, C., Ulrich, M., & Leskovec, J. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186183 · Cited by 173
Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System
Tang, J., & Wang, K. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220021 · Cited by 167
SNR: Sub-Network Routing for Flexible Parameter Sharing in Multi-Task Learning
Ma, J., Zhao, Z., Chen, J., Li, A., Hong, L., & Ed, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301216 · Cited by 165
Self-citationWhy I like it: multi-task learning for recommendation and explanation
Lu, Y., Dong, R., & Smyth, B. (2018) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3240323.3240365 · Cited by 154
Multi-objective optimization for long tail recommendation
Wang, S., Gong, M., Li, H., & Yang, J. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.04.018 · Cited by 152
Related Pins at Pinterest: The Evolution of a Real-World Recommender System
Liu, D. C., Rogers, S., Shiau, R., Kislyuk, D., Kevin, C., Zhong, Z., Liu, J., & Jing, Y. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054202 · Cited by 124
Batch learning from logged bandit feedback through counterfactual risk minimization
SwaminathanAdith, & JoachimsThorsten (2015) · Journal of Machine Learning Research · doi:10.5555/2789272.2886805 · Cited by 110
Estimating Position Bias without Intrusive Interventions
Agarwal, A., Zaitsev, I., Wang, X., Li, C., Najork, M., & Joachims, T. (2019) · Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining · doi:10.1145/3289600.3291017 · Cited by 107
Towards Neural Mixture Recommender for Long Range Dependent User Sequences
Tang, J., Belletti, F., Jain, S., Chen, M., Beutel, A., Xu, C., & Chi, E. H. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313650 · Cited by 101
Self-citationZhai, A., Kislyuk, D., Jing, Y., Feng, M., Tzeng, E., Donahue, J., Du, Y. L., & Darrell, T. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3054201 · Cited by 88
Localized factor models for multi-context recommendation
Agarwal, D., Chen, B.-C., & Long, B. (2011) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2020408.2020504 · Cited by 79
Deconvolving Feedback Loops in Recommender SystemsPreprint
Sinha, A., Gleich, D. F., & Ramani, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01049 · Cited by 63
Practical Lessons from Developing a Large-Scale Recommender System at Zalando
Freno, A. (2017) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3109859.3109897 · Cited by 35
Efficient Training on Very Large Corpora via Gramian EstimationPreprint
Krichene, W., Mayoraz, N., Rendle, S., Zhang, L., Yi, X., Hong, L., Ed, H., & Anderson, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.07187 · Cited by 25
Self-citationA Neural Labeled Network Embedding Approach to Product Adopter Prediction
Gu, Q., Bai, T., Zhao, W. X., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03520-4_8 · Cited by 1
Cited by
Tang, H., Liu, J., Zhao, M., & Gong, X. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3412236 · Cited by 631
Self-supervised Learning for Large-scale Item Recommendations
Yao, T., Yi, X., Cheng, D. Z., Yu, F., Chen, T., Menon, A., Hong, L., Chi, E. H., Tjoa, S., Kang, J. (., et al. (2021) · Proceedings of the 30th ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3481952 · Cited by 253
Self-citationAn Audit of Misinformation Filter Bubbles on YouTube: Bubble Bursting and Recent Behavior Changes
Tomlein, M., Pecher, B., Simko, J., Srba, I., Moro, R., Stefancova, E., Kompan, M., Hrckova, A., Podrouzek, J., & Bielikova, M. (2021) · Fifteenth ACM Conference on Recommender Systems · doi:10.1145/3460231.3474241 · Cited by 65
Auditing YouTube’s Recommendation Algorithm for Misinformation Filter Bubbles
Srba, I., Moro, R., Tomlein, M., Pecher, B., Simko, J., Stefancova, E., Kompan, M., Hrckova, A., Podrouzek, J., Gavornik, A., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3568392 · Cited by 61
Dehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv · doi:10.48550/arxiv.2107.07002 · Cited by 48
Self-citation
Metadata
- Authors
- Zhao, Z., Hong, L., Wei, L., Chen, J., Nath, A., Andrews, S., Kumthekar, A., Sathiamoorthy, M., Yi, X., & Chi, E.
- Venue
- Proceedings of the 13th ACM Conference on Recommender Systems
- Publisher
- ACM
- Published
- 2019-09-10
- Pages
- 43-51
- OpenAlex
- W2973172293
- Cited by
- 311
- References
- 45
- FWCI
- 31.08
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.