Hierarchical Reinforcement Learning for Course Recommendation in MOOCs
Open accessAbstract
The proliferation of massive open online courses (MOOCs) demands an effective way of personalized course recommendation. The recent attention-based recommendation models can distinguish the effects of different historical courses when recommending different target courses. However, when a user has interests in many different courses, the attention mechanism will perform poorly as the effects of the contributing courses are diluted by diverse historical courses. To address such a challenge, we propose a hierarchical reinforcement learning algorithm to revise the user profiles and tune the course recommendation model on the revised profiles.Systematically, we evaluate the proposed model on a real dataset consisting of 1,302 courses, 82,535 users and 458,454 user enrolled behaviors, which were collected from XuetangX—one of the largest MOOCs in China. Experimental results show that the proposed model significantly outperforms the state-of-the-art recommendation models (improving 5.02% to 18.95% in terms of HR@10).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Application scenarios such as smart parking could benefit from recent advances in causal inference and discovery, graph-based reasoning, reinforcement learning, and speech recognition and synthesis.
“These application scenarios could benefit from recent advances in causal inference and discovery [39], graphbased reasoning [40], reinforcement learning [41], and speech recognition and synthesis [24], [42].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [41]
DRL can empower things with the ability to interact with and adapt to the dynamic world, making it useful in many AIoT applications, such as autonomous driving in smart transportation [203], [199], 3D-landmark detection of CT scans [212] and robot control [213] in smart healthcare, course recommendation in smart education [41], real-time scheduling for smart factory [214], load scheduling in smart
“DRL can empower things with the ability to interact with and adapt to the dynamic world, making it useful in many AIoT applications (Figure 9), such as autonomous driving in smart transportation [203], [199], 3D-landmark detection of CT scans [212] and robot control [213] in smart healthcare, course recommendation in smart education [41], real-time scheduling for smart factory [214], load”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [41]
Deep reinforcement learning-based and rule-based recommendation methods have been proposed for personalized learning.
“To this end, deep reinforcement learning-based [41] and rulebased [226] recommendation methods have been proposed.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R. M., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,232 · see citing works
Item-based collaborative filtering recommendation algorithms
Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372071 · Cited by 9,135 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Neural Collaborative Filtering
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052569 · Cited by 7,066 · see citing works
(1999) · Cited by 5,147 · see citing works
BPR: Bayesian Personalized Ranking from Implicit FeedbackPreprint
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 4,604 · see citing works
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119 · see citing works
Factorizing personalized Markov chains for next-basket recommendation
Rendle, S., Freudenthaler, C., & Schmidt-Thieme, L. (2010) · International World Wide Web Conference (WWW) · doi:10.1145/1772690.1772773 · Cited by 2,766 · see citing works
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/239 · Cited by 2,615 · see citing works
Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1402008 · Cited by 2,156 · see citing works
Self-citationSession-based Recommendations with Recurrent Neural NetworksPreprint
Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06939 · Cited by 1,604 · see citing works
Neural Attentive Session-based Recommendation
Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., & Ma, J. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3132926 · Cited by 1,570 · see citing works
Factorization Machines with libFM
Rendle, S. (2012) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2168752.2168771 · Cited by 1,345 · see citing works
The YouTube video recommendation system
Davidson, J., Liebald, B., Liu, J., Nandy, P., Vleet, T. V., Gargi, U., Gupta, S. K., He, Y. H., Lambert, M., Livingston, B., et al. (2010) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1864708.1864770 · Cited by 1,221 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Springer eBooks · doi:10.1007/978-1-4615-3618-5_2 · Cited by 941 · see citing works
Improved Recurrent Neural Networks for Session-based Recommendations
Tan, Y., Xu, X., & Liu, Y. (2016) · Workshop on Deep Learning for Recommender Systems (DLRS) · doi:10.1145/2988450.2988452 · Cited by 810 · see citing works
Two Decades of Recommender Systems at Amazon.com
Smith, B., & Linden, G. (2017) · IEEE Internet Computing · doi:10.1109/mic.2017.72 · Cited by 659 · see citing works
Kabbur, S., Ning, X., & Karypis, G. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487589 · Cited by 651 · see citing works
Reinforcement Learning with Hierarchies of Machines
Parr, R., & Russell, S. (1997) · Cited by 618 · see citing works
NAIS: Neural Attentive Item Similarity Model for Recommendation
He, X., He, Z., Song, J., Liu, Z., Jiang, Y., & Chua, T. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2831682 · Cited by 571 · see citing works
DeepFM: A Factorization-Machine based Neural Network for CTR PredictionPreprint
Guo, H., Tang, R., Ye, Y., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04247 · Cited by 529 · see citing works
Engaging with massive online courses
Anderson, A., Huttenlocher, D. P., Kleinberg, J. M., & Leskovec, J. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2566486.2568042 · Cited by 524 · see citing works
Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks
Huang, J., Zhao, W. X., Dou, H., Wen, J.-R., & Chang, E. Y. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210017 · Cited by 486 · see citing works
Deep Learning over Multi-field Categorical Data
Zhang, W., Du, T., & Wang, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-30671-1_4 · Cited by 465 · see citing works
Reinforcement Learning for Relation Classification From Noisy Data
Feng, J., Huang, M., Li, Z., Yang, Y., & Zhu, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12063 · Cited by 360 · see citing works
Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation
Wang, L., Zhang, W., He, X., & Hongyuan Zha, B. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3219961 · Cited by 236 · see citing works
Reinforcement Learning for Relation Classification from Noisy DataPreprint
Feng, J., Huang, M., Li, Z., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08013 · Cited by 183 · see citing works
Modeling and Predicting Learning Behavior in MOOCs
Qiu, J., Tang, J., Liu, T. X., Gong, J., Zhang, C., Zhang, Q., & Xue, Y. (2016) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2835776.2835842 · Cited by 177 · see citing works
Learning Structured Representation for Text Classification via Reinforcement Learning
Zhang, T., Huang, M., & Li, Z. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12047 · Cited by 150 · see citing works
Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning
Narasimhan, K., Yala, A., & Barzilay, R. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1261 · Cited by 146 · see citing works
Jing, X., & Tang, J. (2017) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3106426.3106478 · Cited by 90 · see citing works
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Cited by 74 · see citing works
Hierarchical policy gradient algorithms
Ghavamzadeh, M., & Mahadevan, S. (2003) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 44 · see citing works
Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension
Wang, Z., Liu, J., Xiao, X., Lyu, Y., & Wu, T. B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1159 · Cited by 41 · see citing works
Bandit Learning with Implicit Feedback
Yi, Q., Wu, Q., Wang, H., Tang, J., & Sun, M. (2018) · neural information processing systems · Cited by 15 · see citing works
Joint Training of Candidate Extraction and Answer Selection for Reading ComprehensionPreprint
Wang, Z., Liu, J., Xiao, X., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06145 · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Kabbur, S.; Ning, X.; and Karypis, G. 2013. Fism: factored item similarity models for top-n recommender systems. In SIGKDD'13, 659-667.
- Tang, J.; Zhang, J.; Yao, L.; Li, J.; Zhang, L.; and Su, Z. 2008. Arnetminer: extraction and mining of academic social networks. In SIGKDD'08, 990-998.
Cited by
Sorted by citations
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774 · see citing works
Self-citationGenerative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 326 · see citing works
Deep reinforcement learning in recommender systems: A survey and new perspectives
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110335 · Cited by 261 · see citing works
Gong, J., Wang, S., Wang, J., Feng, W., Peng, H. M., Tang, J., & Yu, P. S. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401057 · Cited by 190 · see citing works
MOOCCube: A Large-scale Data Repository for NLP Applications in MOOCs
Yu, J., Luo, G., Xiao, T., Zhong, Q., Wang, Y., Feng, W., Luo, J., Wang, C., Hou, L., Li, J., et al. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.285 · Cited by 164 · see citing works
The World is Binary: Contrastive Learning for Denoising Next Basket Recommendation
Qin, Y., Wang, P., & Li, C. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462836 · Cited by 145 · see citing works
Self-citationDiffusion Augmentation for Sequential Recommendation
Liu, Q., Fan, Y. B., Zhao, X., Du, Z., Guo, H., Tang, R., & Tian, F. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615134 · Cited by 137 · see citing works
Amin, S., Uddin, M. I., Alarood, A. A., Mashwani, W. K., Alzahrani, A., & Alzahrani, A. O. (2023) · IEEE Access · doi:10.1109/access.2023.3305584 · Cited by 130 · see citing works
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129 · see citing works
Leveraging Demonstrations for Reinforcement Recommendation Reasoning over Knowledge Graphs
Zhao, K., Wang, X., Zhang, Y., Li, Z., Liu, Z., Xing, C., & Xie, X. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401171 · Cited by 113 · see citing works
Zhu, Y., Lü, H., Qiu, P., Shi, K., Chambua, J., & Niu, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.064 · Cited by 105 · see citing works
A Survey on Reinforcement Learning for Recommender Systems
Lin, Y., Liu, Y., Lin, F., Zou, L., Wu, P., Zeng, W., Chen, H., & Miao, C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3280161 · Cited by 112 · see citing works
Self-citationPOCLib: A High-Performance Framework for Enabling Near Orthogonal Processing on Compression
Zhang, F., Zhai, J., Shen, X., Mutlu, O., & Du, X. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3093234 · Cited by 98 · see citing works
HGNN: Hyperedge-based graph neural network for MOOC Course Recommendation
Wang, X., Ma, W., Guo, L., Jiang, H., Liu, F., & Xu, C. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.102938 · Cited by 91 · see citing works
Layer-refined Graph Convolutional Networks for Recommendation
Zhou, X., Lin, D., Liu, Y., & Miao, C. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00100 · Cited by 91 · see citing works
Adaptive course recommendation in MOOCs
Lin, Y., Feng, S., Lin, F., Zeng, W., Liu, Y., & Wu, P. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107085 · Cited by 83 · see citing works
Hierarchical Reinforcement Learning for Integrated Recommendation
Xie, R., Zhang, S., Wang, R., Xia, F., & Lin, L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16580 · Cited by 80 · see citing works
KGAN: Knowledge Grouping Aggregation Network for course recommendation in MOOCs
Zhang, H., Shen, X., Yi, B., Wang, W., & Feng, Y. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118344 · Cited by 76 · see citing works
EduGraph: Learning Path-Based Hypergraph Neural Networks for MOOC Course Recommendation
Li, M., Li, Z., Huang, C., Jiang, Y., & Wu, X. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3453757 · Cited by 69 · see citing works
Huang, J., Oosterhuis, H., de Rijke, M., & van Hoof, H. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3412252 · Cited by 64 · see citing works
Wei, X., Sun, S., Wu, D., & Zhou, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.767837 · Cited by 59 · see citing works
Equality of Learning Opportunity via Individual Fairness in Personalized Recommendations
Marras, M., Boratto, L., Ramos, G., & Fenu, G. (2021) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00271-1 · Cited by 62 · see citing works
Reinforcement Learning-enhanced Shared-account Cross-domain Sequential Recommendation
Guo, L., Zhang, J., Chen, T., Wang, X., & Yin, H. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3185101 · Cited by 57 · see citing works
Self-citationA Tutorial-Generating Method for Autonomous Online Learning
Wu, X., Wang, H., Zhang, Y., Zou, B., & Hong, H. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3390593 · Cited by 65 · see citing works
A literature review of implemented recommendation techniques used in Massive Open online Courses
Khalid, A., Lundqvist, K. Ø., & Yates, A. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115926 · Cited by 51 · see citing works
Nilashi, M., Bidgoli, B. M., Alghamdi, A., Alrizq, M., Alghamdi, O., Nayer, F. K., Aljehane, N. O., Khosravi, A., & Mohd, S. B. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117092 · Cited by 49 · see citing works
Deep Reinforcement Learning Framework for Category-Based Item Recommendation
Fu, M., Agrawal, A., Irissappane, A. A., Zhang, J., Huang, L., & Qu, H. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3089941 · Cited by 46 · see citing works
Self-citationKnowledge-aware sequence modelling with deep learning for online course recommendation
Deng, W., Zhu, P., Chen, H., Yuan, T., & Wu, J. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103377 · Cited by 45 · see citing works
Self-citationEnabling cross-continent provider fairness in educational recommender systems
Gómez, E., Zhang, C. S., Boratto, L., Salamó, M., & Ramos, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.08.025 · Cited by 44 · see citing works
Zhu, Y., Lin, Q., Lü, H., Shi, K., Liu, D., Chambua, J., Wan, S., & Niu, Z. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3125424 · Cited by 43 · see citing works
Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks
Gong, J., Wan, Y., Liu, Y., Li, X., Zhao, Y., Wang, C., Lin, Y., Fang, X., Feng, W., Zhang, J., et al. (2023) · ACM Transactions on the Web · doi:10.1145/3580510 · Cited by 42 · see citing works
Self-citationContemporary Recommendation Systems on Big Data and Their Applications: A Survey
Xia, Z., Sun, A., Xu, J., Peng, Y., Ma, R., & Cheng, M. (2024) · IEEE Access · doi:10.1109/access.2024.3517492 · Cited by 41 · see citing works
MOOC performance prediction and personal performance improvement via Bayesian network
Hao, J., Gan, J., & Zhu, L. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-10926-8 · Cited by 41 · see citing works
Deep Reinforcement Learning for Sequential Targeting
Wang, W., Li, B., Luo, X., & Wang, X. (2022) · Management Science · doi:10.1287/mnsc.2022.4621 · Cited by 40 · see citing works
Reinforced Explainable Knowledge Concept Recommendation in MOOCs
Jiang, L., Liu, K., Wang, Y., Wang, D., Wang, P., Fu, Y., & Yin, M. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3579991 · Cited by 37 · see citing works
Peng, T., Yuan, H., Zhang, Y., Li, Y., Dai, P., Wang, Q., Wang, S., & Wu, W. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3538706 · Cited by 37 · see citing works
Privacy-preserving federated learning framework in multimedia courses recommendation
Qin, Y., Li, M., & Zhu, J. (2022) · Wireless Networks · doi:10.1007/s11276-021-02854-1 · Cited by 37 · see citing works
Pattern-enhanced Contrastive Policy Learning Network for Sequential Recommendation
Tong, X., Wang, P., Li, C., Xia, L., & Niu, S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/220 · Cited by 36 · see citing works
Self-citationHierarchical reinforcement learning with dynamic recurrent mechanism for course recommendation
Lin, Y., Lin, F., Zeng, W., Xiahou, J., Li, L., Wu, P., Liu, Y., & Miao, C. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108546 · Cited by 36 · see citing works
Personalized Approximate Pareto-Efficient Recommendation
Xie, R., Liu, Y., Zhang, S., Wang, R., Xia, F., & Lin, L. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450039 · Cited by 35 · see citing works
Capacity Tracing-Enhanced Course Recommendation in MOOCs
Tian, X., & Liu, F. (2021) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2021.3083180 · Cited by 34 · see citing works
Finding Paths for Explainable MOOC Recommendation: A Learner Perspective
Frej, J., Shah, N., Knežević, M., Nazaretsky, T., & Käser, T. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636898 · Cited by 39 · see citing works
Zhao, D., Zhang, L., Zhang, B., Zheng, L., Bao, Y., & Yan, W. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401170 · Cited by 32 · see citing works
Contrastive Personalized Exercise Recommendation With Reinforcement Learning
Wu, S., Wang, J., & Zhang, W. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3326449 · Cited by 32 · see citing works
Potential Future Directions in Optimization of Students’ Performance Prediction System
Ahmad, S., ELAffendi, M. A., Anwar, M. S., & Iqbal, R. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/6864955 · Cited by 30 · see citing works
Does Every Data Instance Matter? Enhancing Sequential Recommendation by Eliminating Unreliable Data
Sun, Y., Wang, B., Sun, Z., & Yang, X. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/218 · Cited by 30 · see citing works
Context-aware reinforcement learning for course recommendation
Lin, Y., Lin, F., Yang, L., Zeng, W., Liu, Y., & Wu, P. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109189 · Cited by 30 · see citing works
Ma, H., Huang, Z., Tang, W., Zhu, H., Zhang, H., & Li, J. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3240931 · Cited by 29 · see citing works
DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation
Sun, Z., Fang, H., Yang, J., Qu, X., Liu, H., Yu, D., Ong, Y.-S., & Zhang, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3231891 · Cited by 29 · see citing works
Self-citationLiao, K., Liu, Q., Wei, Z., Peng, B., Chen, Q., Sun, W., & Huang, X. (2020) · arXiv (Cornell University) · Cited by 29 · see citing works
Ma, W., Chen, W., Lu, L., & Fan, X. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103750 · Cited by 33 · see citing works
Multi-knowledge enhanced graph convolution for learning resource recommendation
Dong, Y., Liu, Y., Dong, Y., Wang, Y., & Chen, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111521 · Cited by 27 · see citing works
Li, X., Li, X., Tang, J., Wang, T., Zhang, Y. A., & Chen, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-55130-8_22 · Cited by 26 · see citing works
Self-citationZhao, Z., Yang, Y., Li, C., & Nie, L. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3410441 · Cited by 25 · see citing works
Self-citationGraph path fusion and reinforcement reasoning for recommendation in MOOCs
Liang, Z., Mu, L., Chen, J., & Xie, Q. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11178-2 · Cited by 27 · see citing works
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03540 · Cited by 23 · see citing works
KPCR: Knowledge Graph Enhanced Personalized Course Recommendation
Jung, H., Jang, Y., Kim, S., & Kim, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-97546-3_60 · Cited by 23 · see citing works
A Review of Data Mining in Personalized Education: Current Trends and Future Prospects
Zhang, X., Li, H., Liu, Z., Chen, Z., Zhou, H., Rong, W., & Ouyang, Y. (2024) · Frontiers of digital education. · doi:10.1007/s44366-024-0019-6 · Cited by 23 · see citing works
Hierarchical reinforcement learning for automatic disease diagnosis
Zhong, C., Liao, K., Chen, W., Liu, Q., Peng, B., Huang, X., Peng, J., & Wei, Z. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac408 · Cited by 22 · see citing works
BETA-CD: A Bayesian Meta-Learned Cognitive Diagnosis Framework for Personalized Learning
Bi, H., Chen, E., He, W., Wu, H., Zhao, W., Wang, S., & Wu, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25629 · Cited by 21 · see citing works
Graph Enhanced Hierarchical Reinforcement Learning for Goal-oriented Learning Path Recommendation
Li, Q., Xia, W., Yin, L., Shen, J., Rui, R., Zhang, W., Chen, X., Tang, R., & Yu, Y. H. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614897 · Cited by 25 · see citing works
Sheng, D., Yuan, J., Xie, Q., & Li, L. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.08.001 · Cited by 21 · see citing works
Yang, Z., Ye, J., Wang, L., Lin, X. A., & He, L. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107579 · Cited by 20 · see citing works
Lin, Y., Zhang, W., Lin, F., Zeng, W., Zhou, X., & Wu, P. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09257-7 · Cited by 22 · see citing works
A Study of Deep Reinforcement Learning Based Recommender Systems
Gupta, G., & Katarya, R. (2021) · International Conference on Secure Cyber Computing and Communications (ICSCCC) · doi:10.1109/icsccc51823.2021.9478178 · Cited by 18 · see citing works
HGCR: A Heterogeneous Graph-Enhanced Interactive Course Recommendation Scheme for Online Learning
Wang, Y., Ma, D., Ma, J., & Jin, Q. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3314399 · Cited by 18 · see citing works
Course Recommender Systems Need to Consider the Job Market
Frej, J., Dai, A., Montariol, S., Bosselut, A., & Käser, T. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657847 · Cited by 22 · see citing works
Zhu, J., Chen, Z., Meo, P. D., Guan, J., Han, Z., & Shi, W. (2025) · ACM Transactions on Information Systems · doi:10.1145/3779436 · Cited by 16 · see citing works
Multi-scale reinforced profile for personalized recommendation with deep neural networks in MOOCs
Lin, Y., Feng, S., Lin, F., Xiahou, J., & Zeng, W. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110905 · Cited by 16 · see citing works
Zheng, Z., Sun, Y., Song, X. Y., Zhu, H., & Xiong, H. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3608795 · Cited by 15 · see citing works
Osakwe, I., Chen, G., Fan, Y., Raković, M., Singh, S., Lim, L., van der Graaf, J., Moore, J. D., Molenaar, I., Bannert, M., et al. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13429 · Cited by 17 · see citing works
Ma, J., Zhao, Z., Xie, Z., Zhang, Y., & Zhou, G. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115135 · Cited by 14 · see citing works
A transformer–convolution model for enhanced session-based recommendation
Wang, J., Xie, H., Wang, F. L., & Lee, L. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.01.083 · Cited by 14 · see citing works
Top-N Knowledge Concept Recommendations in MOOCs Using a Neural Co-Attention Model
Klašnja‐Milićević, A., & Milicevic, D. (2023) · IEEE Access · doi:10.1109/access.2023.3278609 · Cited by 14 · see citing works
Knowledge Graph-Based Behavior Denoising and Preference Learning for Sequential Recommendation
Liu, H., Zhu, Y., & Wu, Z. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3325666 · Cited by 14 · see citing works
Zhang, Z., Xiao, Y., Jiang, L., Yang, D., Yin, M., & Wang, P. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i8.28793 · Cited by 14 · see citing works
DIRS-KG: a KG-enhanced interactive recommender system based on deep reinforcement learning
Lin, R., Tang, F., He, C., Wu, Z., Yuan, C., & Tang, Y. Z. (2023) · World Wide Web · doi:10.1007/s11280-022-01135-x · Cited by 14 · see citing works
Attentional Autoencoder for Course Recommendation in MOOC with Course Relevance
Tan, J., Chang, L., Liu, T., & Zhao, X. (2020) · International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) · doi:10.1109/cyberc49757.2020.00038 · Cited by 13 · see citing works
Automatic Generation of Meta-Path Graph for Concept Recommendation in MOOCs
Gong, J., Wang, C., Zhao, Z., & Zhang, X. (2021) · Electronics · doi:10.3390/electronics10141671 · Cited by 13 · see citing works
Zhou, G., Zhao, Z., Ma, J., & Xie, Z. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115809 · Cited by 12 · see citing works
Reinforcement learning-based denoising network for sequential recommendation
Tong, X., Wang, P., & Niu, S. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03298-6 · Cited by 12 · see citing works
Towards Better Course Recommendations: Integrating Multi-Perspective Meta-Paths and Knowledge Graphs
Yang, T. Q., Ren, B., Gu, C., Ma, B., He, T., & Konomi, S. (2025) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3706468.3706486 · Cited by 12 · see citing works
Gu, R. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978241313260 · Cited by 11 · see citing works
Wang, X., Wang, Y.-C., Guo, L., Xu, L., Gao, B., Liu, F., & Li, W. (2021) · Information · doi:10.3390/info12050198 · Cited by 11 · see citing works
Qi, P., Sun, Y., & Luo, H. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22931 · Cited by 11 · see citing works
Narimani, A., & Barberà, E. (2024) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v25i1.7419 · Cited by 10 · see citing works
Personalized Book Recommendation Algorithm for University Library Based on Deep Learning Models
Hou, D. (2022) · Journal of Sensors · doi:10.1155/2022/3087623 · Cited by 10 · see citing works
RetractedMCAP: Low-Pass GNNs with Matrix Completion for Academic Recommendations
Zhang, D., Zheng, S., Zhu, Y., Yuan, H., Gong, J., & Tang, J. (2024) · ACM Transactions on Information Systems · doi:10.1145/3698193 · Cited by 9 · see citing works
Course Recommendation System Based on Course Knowledge Graph Generated by Large Language Models
Chen, X., Yin, C., Chen, H., Rong, W., Ouyang, Y., & Chai, Y. (2024) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale62452.2024.10834324 · Cited by 9 · see citing works
Self-citationSelf‐supervised graph learning for occasional group recommendation
Hao, B., Yin, H., Li, C., & Chen, H. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.23011 · Cited by 9 · see citing works
Self-citationA Systematic Study on Reproducibility of Reinforcement Learning in Recommendation Systems
Cavenaghi, E., Sottocornola, G., Stella, F. A., & Zanker, M. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3596519 · Cited by 9 · see citing works
EduLGCL: Local-global contrastive learning model for education recommendation
Zhao, Y., Jiang, F., Pang, Y., Deng, Y., Han, Y., & Wang, J. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111357 · Cited by 10 · see citing works
An Actor-Critic Hierarchical Reinforcement Learning Model for Course Recommendation
Liang, K., Zhang, G., Guo, J., & Li, W. (2023) · Electronics · doi:10.3390/electronics12244939 · Cited by 8 · see citing works
A Review on Reinforcement Learning based News Recommendation Systems and its challenges
Bangari, S., Nayak, S., Patel, L., & Rashmi, K. T. (2021) · International Conference on Artificial Intelligence and Smart Energy (ICAIS) · doi:10.1109/icais50930.2021.9395812 · Cited by 8 · see citing works
A MOOCs Recommender System Based on User’s Knowledge Background
Zhao, Y., Ma, W., Jiang, Y., & Zhan, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-82136-4_12 · Cited by 8 · see citing works
Du, Z., Yang, N., Yu, Z., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3144292 · Cited by 8 · see citing works
Efficient Course Recommendation using Deep Transformer based Ensembled Attention Model
Madhavi, A., Nagesh, A., & Govardhan, A. (2023) · ICST Transactions on e-Education and e-Learning · doi:10.4108/eetel.4470 · Cited by 7 · see citing works
Item-Difficulty-Aware Learning Path Recommendation: From a Real Walking Perspective
Zhang, H., Shen, S., Xu, B., Huang, Z., Wu, J., Sha, J. J., & Wang, S. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671947 · Cited by 10 · see citing works
Wang, W., Zhou, M., Li, B., & Zhuang, H. (2025) · Information Systems Research · doi:10.1287/isre.2021.0310 · Cited by 7 · see citing works
Personalized Learning Path Recommendation with Time-Aware Attention-Based Reinforcement Learning
Jiang, S., Wen, Y., Shen, J., Peng, G., Kang, G., & Liu, J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3747594 · Cited by 7 · see citing works
- Authors
- Zhang, J., Hao, B., Chen, B., Li, C., Chen, H., & Sun, J.
- Affiliations
- Renmin University of China (CN)
- Georgia Institute of Technology (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 435-442
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2905101140
- Cited by
- 196
- References
- 34
- Citations by year
- 2026: 9 · 2025: 31 · 2024: 33 · 2023: 38 · 2022: 35 · 2021: 31 · 2020: 16 · 2019: 3
- FWCI
- 16.93
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- National Natural Science Foundation of China (61702522,, 61532021, 61772536, 61772537)
- Renmin University of China
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- course recommendation, MOOCs, hierarchical reinforcement learning, personalized recommendation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
