The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Hierarchical Reinforcement Learning for Course Recommendation in MOOCs
Open accessAbstract
The proliferation of massive open online courses (MOOCs) demands an effective way of personalized course recommendation. The recent attention-based recommendation models can distinguish the effects of different historical courses when recommending different target courses. However, when a user has interests in many different courses, the attention mechanism will perform poorly as the effects of the contributing courses are diluted by diverse historical courses. To address such a challenge, we propose a hierarchical reinforcement learning algorithm to revise the user profiles and tune the course recommendation model on the revised profiles.Systematically, we evaluate the proposed model on a real dataset consisting of 1,302 courses, 82,535 users and 458,454 user enrolled behaviors, which were collected from XuetangX—one of the largest MOOCs in China. Experimental results show that the proposed model significantly outperforms the state-of-the-art recommendation models (improving 5.02% to 18.95% in terms of HR@10).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These application scenarios could benefit from recent advances in causal inference and discovery [39], graphbased reasoning [40], reinforcement learning [41], and speech recognition and synthesis [24], [42].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [41]
In the citing paper
“DRL can empower things with the ability to interact with and adapt to the dynamic world, making it useful in many AIoT applications (Figure 9), such as autonomous driving in smart transportation [203], [199], 3D-landmark detection of CT scans [212] and robot control [213] in smart healthcare, course recommendation in smart education [41], real-time scheduling for smart factory [214], load”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [41]
In the citing paper
“To this end, deep reinforcement learning-based [41] and rulebased [226] recommendation methods have been proposed.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R. M., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,232
Item-based collaborative filtering recommendation algorithms
Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372071 · Cited by 9,135
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Neural Collaborative Filtering
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052569 · Cited by 7,385
BPR: Bayesian Personalized Ranking from Implicit FeedbackPreprint
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 6,168
(1999) · Cited by 5,147
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
Factorizing personalized Markov chains for next-basket recommendation
Rendle, S., Freudenthaler, C., & Schmidt-Thieme, L. (2010) · International World Wide Web Conference (WWW) · doi:10.1145/1772690.1772773 · Cited by 2,901
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/239 · Cited by 2,615
ArnetMiner: extraction and mining of academic social networks
Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1402008 · Cited by 2,231
Self-citationNeural Attentive Session-based Recommendation
Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., & Ma, J. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3132926 · Cited by 1,634
Session-based Recommendations with Recurrent Neural NetworksPreprint
Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06939 · Cited by 1,604
Factorization Machines with libFM
Rendle, S. (2012) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2168752.2168771 · Cited by 1,345
The YouTube video recommendation system
Davidson, J., Liebald, B., Liu, J., Nandy, P., Vleet, T. V., Gargi, U., Gupta, S. K., He, Y. H., Lambert, M., Livingston, B., et al. (2010) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1864708.1864770 · Cited by 1,282
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_2 · Cited by 1,024
Improved Recurrent Neural Networks for Session-based Recommendations
Tan, Y., Xu, X., & Liu, Y. (2016) · Workshop on Deep Learning for Recommender Systems (DLRS) · doi:10.1145/2988450.2988452 · Cited by 845
Two Decades of Recommender Systems at Amazon.com
Smith, B., & Linden, G. (2017) · IEEE Internet Computing · doi:10.1109/mic.2017.72 · Cited by 659
Kabbur, S., Ning, X., & Karypis, G. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487589 · Cited by 651
Reinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618
NAIS: Neural Attentive Item Similarity Model for Recommendation
He, X., He, Z., Song, J., Liu, Z., Jiang, Y., & Chua, T. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2831682 · Cited by 571
Engaging with massive online courses
Anderson, A., Huttenlocher, D. P., Kleinberg, J. M., & Leskovec, J. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2566486.2568042 · Cited by 559
Deep Learning over Multi-field Categorical Data
Zhang, W., Du, T., & Wang, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-30671-1_4 · Cited by 545
DeepFM: A Factorization-Machine based Neural Network for CTR PredictionPreprint
Guo, H., Tang, R., Ye, Y., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04247 · Cited by 529
Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks
Huang, J., Zhao, W. X., Dou, H., Wen, J.-R., & Chang, E. Y. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210017 · Cited by 498
Reinforcement Learning for Relation Classification From Noisy Data
Feng, J., Huang, M., Li, Z., Yang, Y., & Zhu, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12063 · Cited by 360
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation
Wang, L., Zhang, W., He, X., & Hongyuan Zha, B. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3219961 · Cited by 236
Reinforcement Learning for Relation Classification from Noisy DataPreprint
Feng, J., Huang, M., Li, Z., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08013 · Cited by 183
Modeling and Predicting Learning Behavior in MOOCs
Qiu, J., Tang, J., Liu, T. X., Gong, J., Zhang, C., Zhang, Q., & Xue, Y. (2016) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2835776.2835842 · Cited by 177
Learning Structured Representation for Text Classification via Reinforcement Learning
Zhang, T., Huang, M., & Li, Z. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12047 · Cited by 153
Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
Narasimhan, K., Yala, A., & Barzilay, R. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1261 · Cited by 146
Jing, X., & Tang, J. (2017) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3106426.3106478 · Cited by 92
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Cited by 74
Hierarchical policy gradient algorithms
Ghavamzadeh, M., & Mahadevan, S. (2003) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 44
Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension
Wang, Z., Liu, J., Xiao, X., Lyu, Y., & Wu, T. B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1159 · Cited by 41
Bandit Learning with Implicit Feedback
Yi, Q., Wu, Q., Wang, H., Tang, J., & Sun, M. (2018) · neural information processing systems · Cited by 15
Joint Training of Candidate Extraction and Answer Selection for Reading ComprehensionPreprint
Wang, Z., Liu, J., Xiao, X., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06145 · Cited by 10
Research works printed but not linked: 1
- Kabbur, S.; Ning, X.; and Karypis, G. 2013. Fism: factored item similarity models for top-n recommender systems. In SIGKDD'13, 659-667.
Cited by
Sorted by citations
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774
Self-citationGenerative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 370
Deep reinforcement learning in recommender systems: A survey and new perspectives
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110335 · Cited by 288
Gong, J., Wang, S., Wang, J., Feng, W., Peng, H. M., Tang, J., & Yu, P. S. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401057 · Cited by 192
MOOCCube: A Large-scale Data Repository for NLP Applications in MOOCs
Yu, J., Luo, G., Xiao, T., Zhong, Q., Wang, Y., Feng, W., Luo, J., Wang, C., Hou, L., Li, J., et al. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.285 · Cited by 172
Metadata
- Authors
- Zhang, J., Hao, B., Chen, B., Li, C., Chen, H., & Sun, J.
- Affiliations
- Renmin University of China (CN)
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 435-442
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2905101140
- Cited by
- 196
- References
- 34
- Citations by year
- 2026: 9 · 2025: 31 · 2024: 33 · 2023: 38 · 2022: 35 · 2021: 31 · 2020: 16 · 2019: 3
- FWCI
- 16.93
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- National Natural Science Foundation of China (61702522,, 61532021, 61772536, 61772537)
- Renmin University of China
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- course recommendation, MOOCs, hierarchical reinforcement learning, personalized recommendation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
