CoChat
Sign Up
Cited by 26Recommender Systems and TechniquesInformation Retrieval and Search BehaviorAdvanced Bandit Algorithms Research

The DOI matches its Crossref record; the references it cites are not checked yet.

Towards Automatic Discovering of Deep Hybrid Network Architecture for Sequential Recommendation

Cheng, Mingyue, Liu, Zhiding, Liu, Qi, Ge, Shenyang, Chen, Enhong·2022·Proceedings of the ACM Web Conference 2022
DOI 10.1145/3485447.3512066
Open this paper in CoChat — free account

Abstract

Recent years have witnessed great success in deep learning-based sequential recommendation (SR), which can provide more timely and accurate recommendations. One of the most effective deep SR architectures is to stack high-performance residual blocks, e.g., prevalent self-attentive and convolutional operations, for capturing long- and short-range dependence of sequential behaviors. By carefully revisiting previous models, we observe: 1) simple architecture modification of gating each residual connection can help us train deeper SR models and yield significant improvements; 2) compared with self-attention mechanism, stacking of convolution layers also can cover each item of the whole sequential behaviors and achieve competitive or even superior performance.

Overview

Integrity signals

Registry

Crossref · 10.1145/3485447.3512066

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The search strategy shifts from discrete reinforcement learning to differentiable searching, which adaptively selects architectures within one-shot learning to circumvent computational burden, for more efficient convergence.

    “The search strategy shifts from the discrete reinforcement learning process, which iteratively samples architectures for training and is time-consuming, into the differentiable searching, which adaptively selects architectures within one-shot learning to circumvent the computational burden, for more efficient convergence.”

    From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [192]

Citations

The citation lists haven’t been fetched yet

References 26 · Cited by 26

Research works printed but not linked: 49

  1. Jimmy Lei Ba Jamie Ryan Kiros and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450(2016). · arXiv
  2. Thomas Bachlechner Bodhisattwa Prasad Majumder Huanru Henry Mao Garrison W Cottrell and Julian McAuley. 2020. Rezero is all you need: Fast convergence at large depth. arXiv preprint arXiv:2003.04887(2020). · arXiv
  3. Philip Bachman R Devon Hjelm and William Buchwalter. 2019. Learning representations by maximizing mutual information across views. In Advances in Neural Information Processing Systems. 15535–15545.
  4. Shaojie Bai J Zico Kolter and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271(2018). · arXiv
  5. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
  6. 10.1109/CVPR46437.2021.01549 · DOI · Paper page
  7. Mingyue Cheng, Runlong Yu, Qi Liu, Vincent W Zheng, Hongke Zhao, Hefu Zhang, and Enhong Chen. 2019. Alpha-Beta Sampling for Pairwise Ranking in One-Class Collaborative Filtering. In 2019 IEEE International Conference on Data Mining (ICDM). IEEE, 1000–1005.
  8. 10.1145/3404835.3462863 · DOI
  9. 10.1109/ICDM51629.2021.00015 · DOI
  10. Stéphane d’Ascoli, Hugo Touvron, Matthew Leavitt, Ari Morcos, Giulio Biroli, and Levent Sagun. 2021. Convit: Improving vision transformers with soft convolutional inductive biases. arXiv preprint arXiv:2103.10697(2021). · arXiv
  11. Alexey et al Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929(2020). · arXiv
  12. Tianyu Gao Xingcheng Yao and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. arXiv preprint arXiv:2104.08821(2021). · arXiv
  13. Jean-Bastien Grill and Florian et al Strub. 2020. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733(2020). · arXiv
  14. 10.1109/CVPR42600.2020.00975 · DOI · Paper page
  15. 10.1145/3077136.3080777 · DOI
  16. Balázs Hidasi Alexandros Karatzoglou Linas Baltrunas and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939(2015). · arXiv
  17. Geoffrey Hinton Oriol Vinyals and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531(2015). · arXiv
  18. 10.1145/3459637.3482483 · DOI · Paper page
  19. Manas R Joglekar Cong Li Mei Chen Taibai Xu Xiaoming Wang Jay K Adams Pranav Khaitan Jiahui Liu and Quoc V Le. 2020. Neural input search for large scale recommendation models. In ACM SIGKDD. 2387–2397. · DOI
  20. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE ICDM. IEEE, 197–206.
  21. Changlin Li Jiefeng Peng Liuchun Yuan Guangrun Wang Xiaodan Liang Liang Lin and Xiaojun Chang. 2020. Block-wisely supervised neural architecture search with knowledge distillation. In CVPR. 1989–1998.
  22. Changlin Li, Tao Tang, and Guangrun et al Wang. 2021. Bossnas: Exploring hybrid cnn-transformers with block-wisely self-supervised neural architecture search. arXiv preprint arXiv:2103.12424(2021). · arXiv
  23. Jing Li Pengjie Ren Zhumin Chen Zhaochun Ren Tao Lian and Jun Ma. 2017. Neural attentive session-based recommendation. In ACM CIKM. 1419–1428.
  24. Zhi Li Hongke Zhao Qi Liu Zhenya Huang Tao Mei and Enhong Chen. 2018. Learning from history and present: Next-item recommendation via discriminatively exploiting user behaviors. In ACM SIGKDD. 1734–1743. · DOI
  25. 10.1145/3394486.3403314 · DOI · Paper page
  26. 10.1007/978-3-030-58548-8_46 · DOI
  27. 10.1109/TSMCB.2011.2163711 · DOI
  28. 10.1109/ICDM.2011.118 · DOI
  29. Renqian Luo Fei Tian Tao Qin Enhong Chen and Tie-Yan Liu. 2018. Neural architecture optimization. NeurIPS 31(2018).
  30. Matthew E. Peters and Mark Neumann et al. 2018. Deep contextualized word representations. In NAACL-HLT.
  31. Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning. PMLR, 4095–4104.
  32. 10.1145/3190616 · DOI
  33. Maithra Raghu Thomas Unterthiner Simon Kornblith Chiyuan Zhang and Alexey Dosovitskiy. 2021. Do Vision Transformers See Like Convolutional Neural Networks?arXiv preprint arXiv:2108.08810(2021). · arXiv
  34. Steffen Rendle Christoph Freudenthaler and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW. 811–820. · DOI
  35. Fei Sun Jun Liu Jian Wu Changhua Pei Xiao Lin Wenwu Ou and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In ACM CIKM. 1441–1450. · DOI · Paper page
  36. 10.1145/3397271.3401125 · DOI
  37. Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In ACM WSDM. 565–573. · DOI
  38. Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. 2018. An empirical study of example forgetting during deep neural network learning. ICLR (2018).
  39. 10.1145/3440207 · DOI
  40. Lechao Xiao and Yasaman et al Bahri. 2018. Dynamical isometry and a mean field theory of cnns: How to train 10 000-layer vanilla convolutional neural networks. In ICML. PMLR 5393–5402.
  41. Jin Xu Xu Tan Renqian Luo Kaitao Song Jian Li Tao Qin and Tie-Yan Liu. 2021. NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search. arXiv preprint arXiv:2105.14444(2021). · DOI
  42. 10.1109/TKDE.2020.3016732 · DOI
  43. 10.1145/3397271.3401156 · DOI
  44. 10.1145/3289600.3290975 · DOI
  45. 10.1145/3437963.3441761 · DOI
  46. 10.1016/j.patcog.2020.107794 · DOI
  47. Xiangyu Zhao, Chong Wang, Ming Chen, Xudong Zheng, Xiaobing Liu, and Jiliang Tang. 2020. Autoemb: Automated embedding dimensionality search in streaming recommendations. arXiv preprint arXiv:2002.11252(2020). · arXiv
  48. 10.1145/3340531.3411954 · DOI
  49. Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578(2016). · arXiv

Metadata

Authors
Cheng, M., Liu, Z., Liu, Q., Ge, S., & Chen, E.
Affiliations
  • University of Science and Technology of China (CN)
  • xiaobing.ai, China
ORCID
Venue
Proceedings of the ACM Web Conference 2022
Publisher
ACM
Published
2022-04-25
Pages
1923-1932
DOI
10.1145/3485447.3512066
OpenAlex
W4224320476
Cited by
26
References
26
Citations by year
2026: 4 · 2025: 6 · 2024: 7 · 2023: 8 · 2022: 1
FWCI
4.92
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
International World Wide Web Conference (WWW) (publishedVersion)
Funders
  • the National Natural Science Foundation of China (61922073, U20A20229)
  • the Youth Innovation Promotion Association of CAS (2014299)
Topics
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Keywords
sequential recommendation, residual blocks, self-attention, convolutional layers, long-range dependence, short-range dependence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review