The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
SSE-PT: Sequential Recommendation Via Personalized Transformer
Open accessAbstract
Temporal information is crucial for recommendation problems because user preferences are naturally dynamic in the real world. Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed for better use of the temporal ordering of items that each user has engaged with. In particular, the SASRec model, inspired by the popular Transformer model in natural languages processing, has achieved state-of-the-art results. However, SASRec, just like the original Transformer model, is inherently an un-personalized model and does not include personalized user embeddings. To overcome this limitation, we propose a Personalized Transformer (SSE-PT) model, outperforming SASRec by almost 5% in terms of [email protected] on 5 real-world datasets. Furthermore, after examining some random users’ engagement history, we find our model not only more interpretable but also able to focus on recent engagement patterns for each user. Moreover, our SSE-PT model with a slight modification, which we call SSE-PT++, can handle extremely long sequences and outperform SASRec in ranking results with comparable training speed, striking a balance between performance and speed requirements. Our novel application of the Stochastic Shared Embeddings (SSE) regularization is essential to the success of personalization. Code and data are open-sourced at https://github.com/wuliwei9278/SSE-PT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
(2014) · Cited by 33,419
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
(2016) · Cited by 11,679
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Item-based collaborative filtering recommendation algorithms
Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372071 · Cited by 9,135 · cited 1×
“SQL-Rank [36] and item-based recommendations [27] are omitted because the former is similar to BPR [25] except using the listwise loss function instead of the pairwise loss function and the latter has been shown inferior to TransRec [10].”
MethodImproving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 1×
“We have confidence to believe that our personalization technique can be similarly applied to Bert4Rec and models with more advanced Bert Backbones (such as Roberta [22], ALBert[20], and Electra [5]) or GPT-3 [3] in future works.”
MethodBPR: Bayesian Personalized Ranking from Implicit Feedback
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 4,242
The MovieLens Datasets: History and Context
Harper, F. M., & Konstan, J. A. (2016) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2827872 · Cited by 3,907
Self-Attentive Sequential Recommendation
Kang, W.-C., & McAuley, J. (2018) · 2018 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2018.00035 · Cited by 3,011
Factorizing personalized Markov chains for next-basket recommendation
Rendle, S., Freudenthaler, C., & Schmidt-Thieme, L. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772773 · Cited by 2,692
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357895 · Cited by 2,558 · cited 4×
“We have confidence to believe that our personalization technique can be similarly applied to Bert4Rec and models with more advanced Bert Backbones (such as Roberta [22], ALBert[20], and Electra [5]) or GPT-3 [3] in future works.”
MethodInfluentialPersonalized Top-N Sequential Recommendation via Convolutional Sequence Embedding
Tang, J., & Wang, K. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159656 · Cited by 1,975
(1992) · Cited by 1,405
Collaborative filtering with temporal dynamics
Koren, Y. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557072 · Cited by 1,203
Maximum-Margin Matrix Factorization
Srebro, N., Rennie, J. D. M., & Jaakkola, T. (2004) · Cited by 962
STAMP: Short-Term Attention/Memory Priority Model for Session-based Recommendation
Liu, Q., Zeng, Y., Mokhosi, R., & Zhang, H. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219950 · Cited by 960
Recurrent Neural Networks with Top-k Gains for Session-based Recommendations
Hidasi, B., & Karatzoglou, A. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271761 · Cited by 925
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
Translation-based Recommendation
He, R., Kang, W.-C., & McAuley, J. (2017) · Proceedings of the Eleventh ACM Conference on Recommender Systems · doi:10.1145/3109859.3109882 · Cited by 412
Multimodal Categorization of Crisis Events in Social Media
Abavisani, M., Wu, L., Hu, S., Tetreault, J., & Jaimes, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01469 · Cited by 188
Self-citationStochastic Shared Embeddings: Data-driven Regularization of Embedding LayersPreprint
Wu, L., Li, S., Hsieh, C., & Sharpnack, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10630 · Cited by 41
Self-citationSQL-Rank: A Listwise Approach to Collaborative RankingPreprint
Wu, L., Hsieh, C., & Sharpnack, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00114 · Cited by 35
Self-citationLarge-scale Collaborative Ranking in Near-Linear Time
Wu, L., Hsieh, C., & Sharpnack, J. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098071 · Cited by 22
Self-citationTemporal Collaborative Ranking Via Personalized TransformerPreprint
Wu, L., Li, S., Hsieh, C., & Sharpnack, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05435 · Cited by 4
Self-citation
Cited by
Wu, L., He, X., Wang, X., Zhang, K., & Wang, M. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3145690 · Cited by 413
Self-citationTransformer Network for Remaining Useful Life Prediction of Lithium-Ion Batteries
Chen, D., Hong, W., & Zhou, X. (2022) · IEEE Access · doi:10.1109/access.2022.3151975 · Cited by 408
Metadata
- Authors
- Wu, L., Li, S., Hsieh, C.-J., & Sharpnack, J.
- Venue
- Fourteenth ACM Conference on Recommender Systems
- Publisher
- ACM
- Published
- 2020-09-22
- Pages
- 328-337
- OpenAlex
- W3005071803
- Cited by
- 260
- References
- 31
- FWCI
- 23.02
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.