CoChat
Sign Up
Cited by 45Recommender Systems and TechniquesInformation Retrieval and Search BehaviorAdvanced Bandit Algorithms Research

AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction

Liu, Bin, Xue, Niannan, Guo, Huifeng, Tang, Ruiming, Zafeiriou, Stefanos, He, Xiuqiang, Li, Zhenguo·2020·Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
DOI 10.1145/3397271.3401082
Open this paper in CoChat — free account

Abstract

Modelling feature interactions is key in Click-Through Rate (CTR) predictions. State-of-the-art models usually include explicit feature interactions to better model non-linearity in a deep network, but enumerating all feature combinations of high orders is not efficient and brings challenges to network optimization. In this work, we use AutoML to seek useful high-order feature interactions to train on without manual feature selection. For this purpose, an end-to-end model, AutoGroup, is proposed, which casts the selection of feature interactions as a structural optimization problem. In a nutshell, AutoGroup first automatically groups useful features into a number of feature sets. Then, it generates interactions of any order from these feature sets using a novel interaction function. The main contribution of AutoGroup is that it performs both dimensionality reduction and feature selection which are not seen in previous models. Offline experiments on three public large-scale benchmark datasets demonstrate the superior performance and efficiency of AutoGroup over state-of-the-art models. Furthermore, a ten-day online A/B test verifies that AutoGroup can be reliably deployed in production and outperform the commercial baseline by 10% on average in terms of CTR and CVR.

Integrity signals

Registry

Crossref · 10.1145/3397271.3401082

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The search strategy shifts from discrete reinforcement learning to differentiable searching, which adaptively selects architectures within one-shot learning to circumvent computational burden, for more efficient convergence.

    “The search strategy shifts from the discrete reinforcement learning process, which iteratively samples architectures for training and is time-consuming, into the differentiable searching, which adaptively selects architectures within one-shot learning to circumvent the computational burden, for more efficient convergence.”

    From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [185]

The citation lists haven’t been fetched yet

References 23 · Cited by 45

Research works printed but not linked: 37

  1. 10.1145/3159652.3159727 · DOI
  2. Mathieu Blondel Akinori Fujino Naonori Ueda and Masakazu Ishihata. 2016. Higher-order factorization machines. In Advances in Neural Information Processing Systems. 3351--3359. Mathieu Blondel Akinori Fujino Naonori Ueda and Masakazu Ishihata. 2016. Higher-order factorization machines. In Advances in Neural Information Processing Systems. 3351--3359.
  3. 10.5555/1756006.1859899 · DOI
  4. 10.1145/2939672.2939785 · DOI · Paper page
  5. Hengtze Cheng , Levent Koc , Jeremiah Harmsen , Tal Shaked , Tushar Deepak Chandra , Hrishi Aradhye , Glen Anderson , Greg S Corrado , Wei Chai , Mustafa Ispir , 2016 . Wide & Deep Learning for Recommender Systems. conference on recommender systems (2016), 7--10. Hengtze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Deepak Chandra, Hrishi Aradhye, Glen Anderson, Greg S Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & Deep Learning for Recommender Systems. conference on recommender systems (2016), 7--10.
  6. 10.1145/2959100.2959190 · DOI · Paper page
  7. Thomas Elsken , Jan Hendrik Metzen, and Frank Hutter . 2018 . Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018). Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2018. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018). · arXiv
  8. Thomas Elsken , Jan Hendrik Metzen, and Frank Hutter . 2018 . Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018). Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2018. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018). · arXiv
  9. Huifeng Guo , Ruiming Tang , Yunming Ye , Zhenguo Li , and Xiuqiang He. 2017. Deep fm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247 ( 2017 ). Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. Deep fm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247 (2017). · arXiv
  10. Xinran He , Junfeng Pan , Ou Jin , Tianbing Xu , Bo Liu , Tao Xu , Yanxin Shi , Antoine Atallah , Ralf Herbrich , and Stuart Bowers .2014. Practical Lessons from Predicting Clicks on Ads at Facebook . In Eighth International Workshop on Data Mining for Online Advertising. 1--9. Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, and Stuart Bowers.2014. Practical Lessons from Predicting Clicks on Ads at Facebook. In Eighth International Workshop on Data Mining for Online Advertising. 1--9.
  11. Kurt Hornik , Maxwell Stinchcombe , and Halbert White . 1989. Multilayer feed forward networks are universal approximators. Neural networks 2, 5 ( 1989 ), 359--366. Kurt Hornik, Maxwell Stinchcombe, and Halbert White. 1989. Multilayer feed forward networks are universal approximators. Neural networks 2, 5 (1989), 359--366.
  12. Eric Jang , Shixiang Gu , and Ben Poole . 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 ( 2016 ). Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016). · arXiv
  13. Yuchin Juan , Yong Zhuang , Wei Sheng Chin , and Chih Jen Lin . 2016 . Field-aware Factorization Machines for CTR Prediction. In ACM Conference on Recommender Systems. 43--50 . Yuchin Juan, Yong Zhuang, Wei Sheng Chin, and Chih Jen Lin. 2016. Field-aware Factorization Machines for CTR Prediction. In ACM Conference on Recommender Systems. 43--50.
  14. Kuang Chih Lee , Burkay Orten , Ali Dasdan , and Wentong Li . 2012 . Estimating conversion rate in display advertising from past performance data . In Acm Sigkdd International Conference on Knowledge Discovery & Data Mining. 768--776 . Kuang Chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past performance data. In Acm Sigkdd International Conference on Knowledge Discovery & Data Mining. 768--776.
  15. Jianxun Lian , Xiaohuan Zhou , Fuzheng Zhang , Zhongxia Chen , Xing Xie , and Guangzhong Sun . 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems. arXiv preprint arXiv:1803.05170 ( 2018 ). Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems. arXiv preprint arXiv:1803.05170 (2018). · arXiv
  16. Andy Liaw , Matthew Wiener, et al . 2002 . Classification and regression by random Forest. R news2, 3 (2002), 18--22. Andy Liaw, Matthew Wiener, et al. 2002. Classification and regression by random Forest. R news2, 3 (2002), 18--22.
  17. Bin Liu Ruiming Tang Yingzhi Chen Jinkai Yu Huifeng Guo and Yuzhou Zhang. 2019. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction. www2019. (2019). https://doi.org/10.1145/3308558.3313497 arXiv:arXiv:1904.04447 Bin Liu Ruiming Tang Yingzhi Chen Jinkai Yu Huifeng Guo and Yuzhou Zhang. 2019. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction. www2019. (2019). https://doi.org/10.1145/3308558.3313497 arXiv:arXiv:1904.04447 · DOI
  18. 10.1007/978-3-030-01246-5_2 · DOI
  19. Hanxiao Liu , Karen Simonyan , and Yiming Yang . 2018 . Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018). Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018). · arXiv
  20. Renqian Luo Fei Tian Tao Qin Enhong Chen and Tie-Yan Liu. 2018. Neural architecture optimization. In Advances in neural information processing systems. 7816--7827. Renqian Luo Fei Tian Tao Qin Enhong Chen and Tie-Yan Liu. 2018. Neural architecture optimization. In Advances in neural information processing systems. 7816--7827.
  21. Chris J Maddison Daniel Tarlow and Tom Minka. 2014. A* sampling. In Advances in Neural Information Processing Systems. 3086--3094. Chris J Maddison Daniel Tarlow and Tom Minka. 2014. A* sampling. In Advances in Neural Information Processing Systems. 3086--3094.
  22. 10.1145/2487575.2488200 · DOI
  23. 10.1145/3178876.3186040 · DOI
  24. Hieu Pham , Melody Y Guan , Barret Zoph , Quoc V Le , and Jeff Dean . 2018. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268 ( 2018 ). Hieu Pham, Melody Y Guan, Barret Zoph, Quoc VLe, and Jeff Dean. 2018. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268 (2018). · arXiv
  25. 10.1109/ICDM.2016.0151 · DOI
  26. Yanru Qu , Bohui Fang , Weinan Zhang , Ruiming Tang , Minzhe Niu , Huifeng Guo , Yong Yu , and Xiuqiang He. 2018. Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data. arXiv preprint arXiv:1807.00311 ( 2018 ). Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data. arXiv preprint arXiv:1807.00311 (2018). · arXiv
  27. 10.1609/aaai.v33i01.33014780 · DOI · Paper page
  28. 10.1109/ICDM.2010.127 · DOI · Paper page
  29. 10.1145/2168752.2168771 · DOI
  30. 10.1145/3124749.3124754 · DOI
  31. 10.1109/CVPR.2019.01099 · DOI
  32. Jun Xiao , Hao Ye , Xiangnan He , Hanwang Zhang , Fei Wu , and Tat-Seng Chua . 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617 ( 2017 ). Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617 (2017). · arXiv
  33. Sirui Xie , Hehui Zheng , Chunxiao Liu , and Liang Lin . 2018. SNAS: stochastic neural architecture search. arXiv preprint arXiv:1812.09926 ( 2018 ). Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. 2018. SNAS: stochastic neural architecture search. arXiv preprint arXiv:1812.09926 (2018). · arXiv
  34. Weinan Zhang Tianming Du and Jun Wang. 2016. Deep Learning over Multifield Categorical Data. (2016). Weinan Zhang Tianming Du and Jun Wang. 2016. Deep Learning over Multifield Categorical Data. (2016). · DOI
  35. Weinan Zhang Tianming Du and Jun Wang. 2016. Deep Learning over Multifield Categorical Data: A Case Study on User Response Prediction. (2016). Weinan Zhang Tianming Du and Jun Wang. 2016. Deep Learning over Multifield Categorical Data: A Case Study on User Response Prediction. (2016). · DOI
  36. Xinbang Zhang , Zehao Huang , and Naiyan Wang . 2018. You only search once: Single shot neural architecture search via direct sparse optimization. arXiv preprint arXiv:1811.01567 ( 2018 ). Xinbang Zhang, Zehao Huang, and Naiyan Wang. 2018. You only search once: Single shot neural architecture search via direct sparse optimization. arXiv preprint arXiv:1811.01567 (2018). · arXiv
  37. 10.1145/3219819.3219823 · DOI · Paper page

Authors
Liu, B., Xue, N., Guo, H., Tang, R., Zafeiriou, S., He, X., & Li, Z.
Affiliations
  • Bytedance & Huawei Noah's Ark Lab, Shenzhen, China
  • Imperial College London (GB)
  • Huawei Technologies (China) (CN)
ORCID
Venue
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
Publisher
ACM
Published
2020-07-25
Pages
199-208
DOI
10.1145/3397271.3401082
OpenAlex
W3034483718
Cited by
45
References
23
Citations by year
2026: 4 · 2025: 5 · 2024: 6 · 2023: 12 · 2022: 6 · 2021: 10 · 2020: 2
FWCI
5.69
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Keywords
click-through rate prediction, feature interaction, high-order feature interactions, automated machine learning, feature selection, dimensionality reduction, feature grouping, interaction function, structural optimization, online A/B testing, conversion rate

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review