The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Ask the GRU: Multi-task Learning for Deep Text Recommendations
Open accessAbstract
In a variety of application domains the content to be recommended to users is associated with text. This includes research papers, movies with associated plot summaries, news articles, blog posts, etc. Recommendation approaches based on latent factor models can be extended naturally to leverage text by employing an explicit mapping from text to factors. This enables recommendations for new, unseen content, and may generalize better, since the factors for all items are produced by a compactly-parametrized model. Previous work has used topic models or averages of word embeddings for this mapping. In this paper we present a method leveraging deep recurrent neural networks to encode the text sequence into a latent vector, specifically gated recurrent units (GRUs) trained end-to-end on the collaborative filtering task. For the task of scientific paper recommendation, this yields models with significantly higher accuracy. In cold-start scenarios, we beat the previous state-of-the-art, all of which ignore word order. Performance is further improved by multi-task learning, where the text encoder network is trained for a combination of content recommendation and item metadata prediction. This regularizes the collaborative filtering model, ameliorating the problem of sparsity of the observed rating matrix.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
(2016) · Cited by 11,679
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,673
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 1×
“We use a bi-directional RNN [52] at the first layer and feed the concatenation of the forward and backward hidden states as the input to the second layer.”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Traditional RNN architectures suffer from the problem of vanishing and exploding gradients [48], rendering optimiza-This is paper about deep learning . tion difficult and prohibiting them from learning long-term dependencies.”
BackgroundCaruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“Existing hybrid methods have used unsupervised learning objectives on text content to regularize the parameters of the recommendation model [4, 23, 24].”
MethodNatural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · cited 1×
“This is particularly important for deep models such as RNNs, since these high-capacity models are prone to overfitting.”
BackgroundProbabilistic Matrix Factorization
Mnih, A., & Salakhutdinov, R. (2007) · UCL Discovery (University College London) · Cited by 4,517
BPR: Bayesian Personalized Ranking from Implicit Feedback
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 4,242
Rendle, S. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.127 · Cited by 3,189
Collaborative Filtering for Implicit Feedback Datasets
Hu, Y., Koren, Y., & Volinsky, C. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.22 · Cited by 3,170
Fab: content-based, collaborative recommendation
Balabanović, M., & Shoham, Y. (1997) · Communications of the ACM · doi:10.1145/245108.245124 · Cited by 2,934
Restricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Methods and metrics for cold-start recommendations
Schein, A. I., Popescul, A., Ungar, L. H., & Pennock, D. M. (2002) · Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/564376.564421 · Cited by 1,780
Collaborative Deep Learning for Recommender Systems
Wang, H., Wang, N., & Yeung, D.-Y. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783273 · Cited by 1,657
Hidden factors and hidden topics: understanding rating dimensions with review text
McAuley, J., & Leskovec, J. (2013) · Proceedings of the 7th ACM conference on Recommender systems · doi:10.1145/2507157.2507163 · Cited by 1,564
Collaborative topic modeling for recommending scientific articles
Wang, C., & Blei, D. M. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020480 · Cited by 1,527
An Empirical Exploration of Recurrent Network Architectures
Józefowicz, R., Zaremba, W., & Sutskever, I. (2015) · International Conference on Machine Learning · Cited by 1,459
Deep content-based music recommendation
van den Oord, A., Dieleman, S., & Schrauwen, B. (2013) · Ghent University Academic Bibliography (Ghent University) · Cited by 1,426
SoRec: social recommendation using probabilistic matrix factorization
Ma, H., Yang, H., Lyu, M. R., & King, I. (2008) · Proceedings of the 17th ACM conference on Information and knowledge management · doi:10.1145/1458082.1458205 · Cited by 1,401
Content-based book recommending using learning for text categorization
Mooney, R. J., & Roy, L. (2000) · Proceedings of the fifth ACM conference on Digital libraries · doi:10.1145/336597.336662 · Cited by 1,328
Relational learning via collective matrix factorization
Singh, A. P., & Gordon, G. J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401969 · Cited by 1,204
AutoRec: Autoencoders Meet Collaborative Filtering
Sedhain, S., Menon, A. K., Sanner, S., & Xie, L. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742726 · Cited by 1,175
Content-boosted collaborative filtering for improved recommendations
Melville, P., Mooney, R. J., & Nagarajan, R. (2002) · doi:10.5555/777092.777124 · Cited by 1,084
Topic modeling: beyond bag-of-words
Wallach, H. M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143967 · Cited by 1,058
Collaborative Denoising Auto-Encoders for Top-N Recommender Systems
Wu, Y., DuBois, C., Zheng, A. X., & Ester, M. (2016) · Proceedings of the Ninth ACM International Conference on Web Search and Data Mining · doi:10.1145/2835776.2835837 · Cited by 1,001
VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback
He, R., & McAuley, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9973 · Cited by 992
Recommendation as classification: using social and content-based information in recommendation
Basu, C., Hirsh, H., & Cohen, W. W. (1998) · Cited by 955
Shi, Y., Larson, M., & Hanjalic, A. (2014) · ACM Computing Surveys · doi:10.1145/2556270 · Cited by 806
Lessons from the Netflix prize challenge
Bell, R. M., & Koren, Y. (2007) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1345448.1345465 · Cited by 727
A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems
Elkahky, A. M., Song, Y., & He, X. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741667 · Cited by 704
Wsabie: Scaling up to large vocabulary image annotation
Weston, J., Samy Bengio, A., & Usunier, N. (2011) · Cited by 677
Visualizing and Understanding Neural Models in NLP
Li, J., Chen, X., Hovy, E., & Jurafsky, D. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1082 · Cited by 625
Regression-based latent factor models
Agarwal, D., & Chen, B.-C. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557029 · Cited by 514
Latent dirichlet allocation for tag recommendation
Krestel, R., Fankhauser, P., & Nejdl, W. (2009) · Proceedings of the third ACM conference on Recommender systems · doi:10.1145/1639714.1639726 · Cited by 494
Using twitter to recommend real-time topical news
Phelan, O., McCarthy, K., & Smyth, B. (2009) · Proceedings of the third ACM conference on Recommender systems · doi:10.1145/1639714.1639794 · Cited by 419
Improving Content-based and Hybrid Music Recommendation using Deep Learning
Wang, X., & Wang, Y. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654940 · Cited by 367
Unifying collaborative and content-based filtering
Basilico, J., & Hofmann, T. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015394 · Cited by 352
Ratings meet reviews, a combined approach to recommend
Ling, G., Lyu, M. R., & King, I. (2014) · Proceedings of the 8th ACM Conference on Recommender systems · doi:10.1145/2645710.2645728 · Cited by 348
Social media recommendation based on people and tags
Guy, I., Zwerdling, N., Ronen, I., Carmel, D., & Uziel, E. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835484 · Cited by 312
Natural Language Processing (Almost) from Scratch
CollobertRonan, WestonJason, BottouLéon, KarlenMichael, KavukcuogluKoray, & KuksaPavel (2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078186 · Cited by 306
Learning Attribute-to-Feature Mappings for Cold-Start Recommendations
Gantner, Z., Drumond, L., Freudenthaler, C., Rendle, S., & Schmidt-Thieme, L. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.129 · Cited by 289
Neural Network Matrix FactorizationPreprint
Dziugaite, G. K., & Roy, D. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06443 · Cited by 216
#TagSpace: Semantic Embeddings from Hashtags
Weston, J., Chopra, S., & Adams, K. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1194 · Cited by 166
Content-based recommendations with Poisson factorization
Gopalan, P., Charlin, L., & Blei, D. M. (2014) · neural information processing systems · Cited by 158
Extraction of Salient Sentences from Labelled DocumentsPreprint
Denil, M., Demiraj, A., & de Freitas, N. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6815 · Cited by 138
Learning Distributed Representations from Reviews for Collaborative Filtering
Almahairi, A., Kastner, K., Cho, K., & Courville, A. C. (2015) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2792838.2800192 · Cited by 110
Joint Training of Deep Boltzmann MachinesPreprint
Goodfellow, I., Courville, A. C., & Bengio, Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1212.2686 · Cited by 25
Cited by
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219890 · Cited by 3,733
Deep Learning Based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2020) · ACM Computing Surveys · doi:10.1145/3285029 · Cited by 3,485
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220007 · Cited by 1,233
Deep Learning based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07435 · Cited by 635
Tang, H., Liu, J., Zhao, M., & Gong, X. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3412236 · Cited by 631
Metadata
- Authors
- Bansal, T., Belanger, D., & McCallum, A.
- Venue
- Proceedings of the 10th ACM Conference on Recommender Systems
- Publisher
- ACM
- Published
- 2016-09-07
- Pages
- 107-114
- OpenAlex
- W2510317721
- Cited by
- 306
- References
- 56
- FWCI
- 29.92
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.