The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
DRN: A Deep Reinforcement Learning Framework for News Recommendation
Open accessAbstract
In this paper, we propose a novel Deep Reinforcement Learning framework for news recommendation. Online personalized news recommendation is a highly challenging problem due to the dynamic nature of news features and user preferences. Although some online recommendation models have been proposed to address the dynamic nature of news recommendation, these methods have three major issues. First, they only try to model current reward (e.g., Click Through Rate). Second, very few studies consider to use user feedback other than click / no click labels (e.g., how frequent user returns) to help improve recommendation. Third, these methods tend to keep recommending similar news to users, which may cause users to get bored. Therefore, to address the aforementioned challenges, we propose a Deep Q-Learning based recommendation framework, which can model future reward explicitly. We further consider user return pattern as a supplement to click / no click label in order to capture more user feedback information. In addition, an effective exploration strategy is incorporated to find new attractive news for users. Extensive experiments are conducted on the offline dataset and online production environment of a commercial news recommendation application and have shown the superior performance of our methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Adomavicius, G., & Tuzhilin, A. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.99 · Cited by 10,310
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Wide & Deep Learning for Recommender Systems
Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., et al. (2016) · Proceedings of the 1st Workshop on Deep Learning for Recommender Systems · doi:10.1145/2988450.2988454 · Cited by 3,535
Miller, R. G. (1981) · Cited by 3,336
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Rendle, S. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.127 · Cited by 3,189
Bobadilla, J., Ortega, F., Hernando, A., & Gutiérrez, A. (2013) · Knowledge-Based Systems · doi:10.1016/j.knosys.2013.03.012 · Cited by 3,158
A contextual-bandit approach to personalized news article recommendation
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772758 · Cited by 2,562
Improving recommendation lists through topic diversification
Ziegler, C.-N., McNee, S. M., Konstan, J. A., & Lausen, G. (2005) · Proceedings of the 14th international conference on World Wide Web - WWW '05 · doi:10.1145/1060745.1060754 · Cited by 1,892
Google news personalization: scalable online collaborative filtering
Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242610 · Cited by 1,511
Reinforcement Learning: An Introduction
(1998) · Kybernetes · doi:10.1108/k.1998.27.9.1093.3 · Cited by 1,472
Joint Deep Modeling of Users and Items Using Reviews for Recommendation
Zheng, L., Noroozi, V., & Yu, P. S. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018665 · Cited by 1,075
(2011) · Cited by 1,000
Personalized news recommendation based on click behavior
Liu, J., Dolan, P., & Pedersen, E. R. (2010) · Proceedings of the 15th international conference on Intelligent user interfaces · doi:10.1145/1719970.1719976 · Cited by 762
Improving Aggregate Recommendation Diversity Using Ranking-Based Techniques
Adomavicius, G., & Kwon, Y. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.15 · Cited by 675
A Theoretical Analysis of NDCG Type Ranking Measures
Wang, Y., Wang, L., Li, Y., He, D., Liu, T.-Y., & Chen, W. (2013) · arXiv · doi:10.48550/arxiv.1304.6480 · Cited by 351
Interactively optimizing information retrieval systems as a dueling bandits problem
Yue, Y., & Joachims, T. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553527 · Cited by 268
SCENE: a scalable two-stage personalized news recommendation system
Li, L., Wang, D., Li, T., Knox, D., & Padmanabhan, B. (2011) · Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval · doi:10.1145/2009916.2009937 · Cited by 213
Online Clustering of BanditsPreprint
Gentile, C., Li, S., & Zappella, G. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.8257 · Cited by 177
Dynamic Attention Deep Model for Article Recommendation by Learning Human Editors' Demonstration
Wang, X., Yu, L., Ren, K., Tao, G., Zhang, W., Yu, Y. H., & Wang, Y.-X. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098096 · Cited by 172
From chatter to headlines: harnessing the real-time web for personalized news recommendation
Morales, G. D. F., Gionis, A., & Lucchese, C. (2012) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2124295.2124315 · Cited by 168
Interactive collaborative filtering
Zhao, X., Zhang, W., & Wang, J. (2013) · Proceedings of the 22nd ACM international conference on Information & Knowledge Management · doi:10.1145/2505515.2505690 · Cited by 165 · cited 1×
“Recently, some people try to combine bandit with clustering based collaborative filtering [14], andmatrix factorization [6, 21, 32, 42, 43, 51], in order to model more complex user and item relationship, and utilize the social network relationship in determining the reward function.”
BackgroundJing, H., & Smola, A. J. (2017) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3018661.3018719 · Cited by 161
Online Context-Aware Recommendation with Time Varying Multi-Armed Bandit
Zeng, C., Wang, Q., Mokhtari, S., & Li, T. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939878 · Cited by 140
Exploration in Interactive Personalized Music Recommendation: A Reinforcement Learning Approach
Wang, X., Wang, Y., Hsu, D., & Wang, Y. (2014) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/2623372 · Cited by 137
Reusing historical interaction data for faster online learning to rank for IR
Hofmann, K., Schuth, A., Whiteson, S., & de Rijke, M. (2013) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2433396.2433419 · Cited by 130
Learning and adaptivity in interactive recommender systems
Mahmood, T., & Ricci⋆, F. (2007) · International Conference on Electronic Commerce · doi:10.1145/1282100.1282114 · Cited by 127
Content-Based News Recommendation
Kompan, M., & Bieliková, M. (2010) · Lecture notes in business information processing · doi:10.1007/978-3-642-15208-5_6 · Cited by 126
A Contextual-Bandit Algorithm for Mobile Context-Aware Recommender System
Bouneffouf, D., Bouzeghoub, A., & Gançarski, A. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34487-9_40 · Cited by 121
Efficient Thompson sampling for online matrix-factorization recommendation
Kawale, J., Bui, H., Kveton, B., Thanh, L. T., & Chawla, S. (2015) · ePrints Soton (University of Southampton) · doi:10.5555/2969239.2969384 · Cited by 115
Ibrahim, J. G., Chen, M., & Sinha, D. (2005) · Encyclopedia of Biostatistics · doi:10.1002/0470011815.b2a11006 · Cited by 112
Factorization Bandits for Interactive Recommendation
Wang, H., Wu, Q., & Wang, H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10936 · Cited by 103
Terms of a Feather: Content-Based News Recommendation and Discovery Using Twitter
Phelan, O., McCarthy, K., Bennett, M., & Smyth, B. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-20161-5_44 · Cited by 100
Ensemble contextual bandits for personalized recommendation
Tang, L., Jiang, Y., Li, L., & Li, T. (2014) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2645710.2645732 · Cited by 100
Learning Hidden Features for Contextual Bandits
Wang, H., Wu, Q., & Wang, H. (2016) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2983323.2983847 · Cited by 93
Returning is Believing: Optimizing Long-term User Engagement in Recommender Systems
Wu, Q., Wang, H., Hong, L., & Shi, Y. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133025 · Cited by 92
The multiple multiplicative factor model for collaborative filtering
Marlin, B. M., & Zemel, R. S. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015437 · Cited by 87
Usage-based web recommendations
Taghipour, N., Kardan, A. A., & Ghidary, S. S. (2007) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1297231.1297250 · Cited by 84
Personalized Recommendation via Parameter-Free Contextual Bandits
Tang, L., Jiang, Y., Li, L., Zeng, C., & Li, T. (2015) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2766462.2767707 · Cited by 79
Online Learning to Rank for Information Retrieval: SIGIR 2016 Tutorial
Grotov, A., & de Rijke, M. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2914798 · Cited by 48
A UCB-Like Strategy of Collaborative Filtering
Nakamura, A. (2014) · Asian Conference on Machine Learning · Cited by 11
Cited by
Deep Learning Based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2020) · ACM Computing Surveys · doi:10.1145/3285029 · Cited by 3,485
Knowledge Graph Convolutional Networks for Recommender Systems
Wang, H., Zhao, M., Xie, X., Li, W., & Guo, M. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313417 · Cited by 1,021
Self-citationDeep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
“nfidence Bounds (UCB). Some work estimates action-value reward function Q(s;a) with Q network using the Bellman equation and finds the best strategy with the largest function value [148], [149], [150], [151]. Also, the actor network has been adopted recently to learn the best policy by maximizing the long term reward [1…”
MethodDeep Learning based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07435 · Cited by 635
Metadata
- Authors
- Zheng, G., Zhang, F., Zheng, Z., Xiang, Y., Yuan, N. J., Xie, X., & Li, Z.
- Venue
- Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18
- Publisher
- ACM Press
- Published
- 2018
- Pages
- 167-176
- OpenAlex
- W2787933113
- Cited by
- 630
- References
- 44
- FWCI
- 55.41
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.