One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction
The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Traditional industry recommendation systems usually use data in a single domain to train models and then serve the domain. However, a large-scale commercial platform often contains multiple domains, and its recommendation system often needs to make click-through rate (CTR) predictions for multiple domains. Generally, different domains may share some common user groups and items, and each domain may have its own unique user groups and items. Moreover, even the same user may have different behaviors in different domains. In order to leverage all the data from different domains, a single model can be trained to serve all domains. However, it is difficult for a single model to capture the characteristics of various domains and serve all domains well. On the other hand, training an individual model for each domain separately does not fully use the data from all domains. In this paper, we propose the Star Topology Adaptive Recommender (STAR) model to train a single model to serve all domains by leveraging data from all domains simultaneously, capturing the characteristics of each domain, and modeling the commonalities between different domains. Essentially, the net- work of each domain consists of two factorized networks: one centered network shared by all domains and the domain-specific network tailored for each domain. For each domain, we combine these two factorized networks and generate a unified network by element-wise multiplying the weights of the shared network and those of the domain-specific network, although these two factorized networks can be combined using other functions, which is open for further research. Most importantly, STAR can learn the shared network from all the data and adapt domain-specific parameters according to the characteristics of each domain. The experimental results from production data validate the superiority of the proposed STAR model. Since late 2020, STAR has been deployed in the display advertising system of Alibaba, obtaining 8.0% improvement on CTR and 6.0% increase on RPM (Revenue Per Mille).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 1×
“Inspired by the success within deep learning, recent CTR prediction model has made the transition from traditional shallow approaches [11, 19, 20, 32, 44] to modern deep approaches [6, 13, 28, 30, 42, 43].”
BackgroundBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · cited 1×
“Among all normalization methods, batch normalization (BN) [14] is a representative method that is proved to be crucial to the successful training of very deep neural networks [14, 31].”
MethodMatrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,673 · cited 1×
“Inspired by the success within deep learning, recent CTR prediction model has made the transition from traditional shallow approaches [11, 19, 20, 32, 45] to modern deep approaches [6, 13, 28, 30, 43, 44].”
BackgroundA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Factorization meets the neighborhood: a multifaceted collaborative filtering model
Koren, Y. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401944 · Cited by 3,852
Wide & Deep Learning for Recommender Systems
Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., et al. (2016) · Proceedings of the 1st Workshop on Deep Learning for Recommender Systems · doi:10.1145/2988450.2988454 · Cited by 3,535 · cited 4×
“Wide & Deep [6] and deepFM [13] combine low-order and high-order features to improve the expression power of the model.”
BackgroundInfluentialA theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-009-5152-4 · Cited by 3,515 · cited 1×
“As such, it contrasts with the domain adaptation (DA) problem [3, 4], where knowledge transfer is only one way, i.”
BackgroundOn calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · cited 1×
“In CPC, the CTR model needs to be well-calibrated [12] in order to achieve a competitive advertising system, i.”
MethodRendle, S. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.127 · Cited by 3,189 · cited 2×
“Inspired by the success within deep learning, recent CTR prediction model has made the transition from traditional shallow approaches [11, 19, 20, 32, 45] to modern deep approaches [6, 13, 28, 30, 43, 44].”
BackgroundMulti-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00781 · Cited by 3,151
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Guo, H., TANG, R., Ye, Y., Li, Z., & He, X. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/239 · Cited by 2,538 · cited 4×
“Wide & Deep [6] and deepFM [13] combine low-order and high-order features to improve the expression power of the model.”
BackgroundInfluentialAn Overview of Multi-Task Learning in Deep Neural Networks
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 2,265 · cited 2×
“One possible strategy to improve learning with multiple domains is multi-task learning [5, 25, 33].”
BackgroundDeep Interest Network for Click-Through Rate Prediction
Zhou, G., Zhu, X., Song, C., Fan, Y., Zhu, H., Ma, X., Yan, Y., Jin, J., Li, H., & Gai, K. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219823 · Cited by 2,128 · cited 13×
“Although the Shared Bottom, MMoE, and Cross-Stitch achieve better overall performance than the Base model, it is notable that in some domains, the AUCs of Shared Bottom, MMoE, and CrossStitch are worse than the Base model, e.g., domain # 5, #6, and #16.”
BackgroundMethodInfluentialSelf-citationProceedings for the 5th International Conference on Learning Representations
Srivastava, A., & Sutton, C. A. (2017) · Cited by 1,837
Convex multi-task feature learning
Argyriou, A., Evgeniou, T., & Pontil, M. (2008) · Machine Learning · doi:10.1007/s10994-007-5040-8 · Cited by 1,528 · cited 1×
“[1] propose a method to learn sparse representations shared across multiple tasks.”
MethodCross-Stitch Networks for Multi-task Learning
Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.433 · Cited by 1,397 · cited 2×
“Cross-Stitch [27] uses linear cross-stitch units to learn an optimal combination of task-specific representations.”
BackgroundMethodModeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220007 · Cited by 1,233 · cited 3×
“MMoE [25] implicitly models task relationships for multi-task learning, where different tasks may have different label spaces.”
BackgroundMethodDeng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178 · cited 1×
“Multi-task learning has been used successfully on multiple application domains, ranging from natural language processing [7], speech recognition [8], recommender system [42] to computer vision [17].”
MethodCaruana, R. (1998) · Cited by 1,110
Deep Interest Evolution Network for Click-Through Rate Prediction
Zhou, G., Mou, N., Fan, Y., Pi, Q., Bian, W., Zhou, C., Zhu, X., & Gai, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015941 · Cited by 1,099
Self-citationxDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., & Sun, G. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220023 · Cited by 1,040
Large-Scale Parallel Collaborative Filtering for the Netflix Prize
Zhou, Y., Wilkinson, D., Schreiber, R., & Pan, R. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-68880-8_32 · Cited by 700
Product-Based Neural Networks for User Response Prediction
Qu, Y., Cai, H., Ren, K., Zhang, W., Yu, Y., Wen, Y., & Wang, J. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0151 · Cited by 670 · cited 2×
“PNN [30] introduces a product layer to capture interactive patterns between inter-field categories.”
BackgroundTang, H., Liu, J., Zhao, M., & Gong, X. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3412236 · Cited by 631
Ba, J. L., Kiros, J., & Hinton, G. E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06450 · Cited by 440
Discriminative learning for differing training and test distributions
Bickel, S., Brückner, M., & Scheffer, T. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273507 · Cited by 387 · cited 1×
“As such, it contrasts with the domain adaptation (DA) problem [3, 4], where knowledge transfer is only one way, i.”
BackgroundEntire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate
Ma, X., Zhao, L., Huang, G., Wang, Z., Hu, Z., Zhu, X., & Gai, K. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210104 · Cited by 382 · cited 2×
“, jointly estimate CTR and conversion rate (CVR) [26, 38].”
BackgroundSelf-citationMulti-Interest Network with Dynamic Routing for Recommendation at Tmall
Li, C., Liu, Z., Wu, M., Xu, Y., Zhao, H., Huang, P., Kang, G., Chen, Q., Li, W., & Lee, D. L. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357814 · Cited by 380 · cited 2×
“MIND [21] and DMIN [39] argue that a single vector might be insufficient to capture complicated pattern lying in the user and items.”
BackgroundDeep Session Interest Network for Click-Through Rate Prediction
Feng, Y., Lv, F., Shen, W., Wang, M., Sun, F., Zhu, Y., & Yang, K. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/319 · Cited by 295
Pi, Q., Zhou, G., Zhang, Y., Wang, Z., Ren, L., Fan, Y., Zhu, X., & Gai, K. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412744 · Cited by 264 · cited 1×
“SIM [29] extracts user interests with two cascaded search units, which achieves better ability to model lifelong sequential behavior data in both scalability and accuracy.”
BackgroundSelf-citationPractice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction
Pi, Q., Bian, W., Zhou, G., Zhu, X., & Gai, K. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330666 · Cited by 216 · cited 2×
“MIMN [28] proposes a memorybased architecture to aggregate features and tackle the challenge of long-term user interest modeling.”
BackgroundSelf-citationWen, H., Zhang, J., Wang, Y., Lv, F., Bao, W., Lin, Q., & Yang, K. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401443 · Cited by 192
Optimized Cost per Click in Taobao Display Advertising
Zhu, H., Jin, J., Tan, C., Pan, F., Zeng, Y., Li, H., & Gai, K. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098134 · Cited by 187 · cited 1×
“domains range from Guess What You Like in Taobao App homepage, Banner of Taobao App homepage to other little data domains [45].”
BackgroundSNR: Sub-Network Routing for Flexible Parameter Sharing in Multi-Task Learning
Ma, J., Zhao, Z., Chen, J., Li, A., Hong, L., & Ed, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301216 · Cited by 164 · cited 1×
“By contrast, most multi-task learning approaches [24–27, 36] address To fully exploit the domain relationship, we propose Star Topology Adaptive Recommender ( STAR ) for multi-domain CTR prediction.”
BackgroundDeep Multi-Interest Network for Click-through Rate Prediction
Xiao, Z., Yang, L., Jiang, W., Wei, Y., Hu, Y., & Wang, H. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412092 · Cited by 152 · cited 1×
“MIND [21] and DMIN [39] argue that a single vector might be insufficient to capture complicated pattern lying in the user and items.”
BackgroundLi, P., Li, R., Da, Q., Zeng, A., & Zhang, L. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412713 · Cited by 131 · cited 1×
“In real-world applications, it is oftentimes that the data are collected from multiple domains [9, 16, 22, 41].”
BackgroundMulti-domain learning by confidence-weighted parameter combination
Dredze, M. H., Kulesza, A., & Crammer, K. (2009) · Machine Learning · doi:10.1007/s10994-009-5148-0 · Cited by 121 · cited 1×
“In real-world applications, it is oftentimes that the data are collected from multiple domains [9, 16, 22, 41].”
BackgroundXDL: an industrial deep learning framework for high-dimensional sparse data
Jiang, B., Deng, C., Yi, H., Hu, Z., Zhou, G., Yang, Z., Huang, S., Guo, X., Wang, D., Song, Y., et al. (2019) · doi:10.1145/3326937.3341255 · Cited by 71
Self-citationMulti-Domain Learning: When Do Domains Matter?
Joshi, M., Dredze, M. H., & Cohen, W. W. (2013) · Cited by 50
One Person, One Model, One World: Learning Continual User Representation without ForgettingPreprint
Yuan, F., Zhang, G., Karatzoglou, A., Jose, J. M., Kong, B., & Li, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13724 · Cited by 16 · cited 1×
“Multi-task learning has been used successfully on multiple application domains, ranging from natural language processing [7], speech recognition [8], recommender system [42] to computer vision [17].”
Method
Metadata
- Authors
- Sheng, X.-R., Zhao, L., Zhou, G., Ding, X., Dai, B., Luo, Q., Yang, S., Lv, J., Zhang, C., Deng, H., & Zhu, X.
- Venue
- Proceedings of the 30th ACM International Conference on Information & Knowledge Management
- Publisher
- ACM
- Published
- 2021-10-26
- Pages
- 4104-4113
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W3209943551
- Cited by
- 197
- References
- 43
- FWCI
- 26.68
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.