The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Generative Adversarial User Model for Reinforcement Learning Based Recommendation System
Abstract
There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a novel model-based reinforcement learning framework for recommendation systems, where we develop a generative adversarial network to imitate user behavior dynamics and learn her reward function. Using this user model as the simulation environment, we develop a novel Cascading DQN algorithm to obtain a combinatorial recommendation policy which can handle a large number of candidate items efficiently. In our experiments with real data, we show this generative adversarial user model can better explain user behavior than alternatives, and the RL policy based on this model can lead to a better long-term reward for the user and higher click rate for the system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Building upon this capability, there has been a growing research area that employs LLMs as central controllers to construct autonomous agents to obtain human-like decisionmaking capabilities [11][12][13][14][15][16][17].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Research works printed but not linked: 25
- Abbeel, P. and Ng, A. Y. Apprenticeship learning via inverse reinforcement learning. In ICML, 2004.
- Chen, T. and Guestrin, C. Xgboost: A scalable tree boost- ing system. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785-794. ACM, 2016.
- Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., Anil, R., Haque, Z., Hong, L., Jain, V., Liu, X., and Shah, H. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016.
- Clavera, I., Nagabandi, A., Fearing, R. S., Abbeel, P., Levine, S., and Finn, C. Learning to adapt: Meta- learning for model-based control. arXiv preprint arXiv:1803.11347, 2018. · arXiv
- Deisenroth, M. P., Fox, D., and Rasmussen, C. E. Gaussian processes for data-efficient learning in robotics and con- trol. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015. ISSN 0162-8828.
- Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta- learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400, 2017. · arXiv
- Guo, H., Tang, R., Ye, Y., Li, Z., and He, X. Deepfm: a factorization-machine based neural network for ctr pre- diction. arXiv preprint arXiv:1703.04247, 2017. · arXiv
- He, J., Ostendorf, M., He, X., Chen, J., Gao, J., Li, L., and Deng, L. Deep reinforcement learning with a combinato- rial action space for predicting popular reddit threads. In EMNLP, 2016.
- Hidasi, B., Karatzoglou, A., Baltrunas, L., and Tikk, D. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939, 2015. · arXiv
- Hidasi, B., Karatzoglou, A., Baltrunas, L., and Tikk, D. Session-based recommendations with recurrent neural networks. In ICLR, 2016.
- Ho, J. and Ermon, S. Generative adversarial imitation learn- ing. In NIPS, 2016.
- Ho, J., Gupta, J. K., and Ermon, S. Model-free imitation learning with policy optimization. In ICML, 2016.
- Jannach, D. and Ludewig, M. When recurrent neural net- works meet the neighborhood for session-based recom- mendation. In Proceedings of the Eleventh ACM Con- ference on Recommender Systems, pp. 306-310. ACM, 2017.
- Li, L., Chu, W., Langford, J., and Schapire, R. E. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th interna- tional conference on World wide web, pp. 661-670. ACM, 2010.
- Manski, C. F. Maximum score estimation of the stochastic utility model of choice. Journal of Econometrics, pp. 205 -228, 1975. ISSN 0304-4076.
- Mataric, M. J. Reward functions for accelerated learning. In ICML, 1994.
- McFadden, D. Conditional logit analysis of qualitative choice behaviour. In Zarembka, P. (ed.), Frontiers in Econometrics, pp. 105-142. Academic Press New York, 1973.
- Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. · arXiv
- Nagabandi, A., Kahn, G., Fearing, R. S., and Levine, S. Neural network dynamics for model-based deep rein- forcement learning with model-free fine-tuning. arXiv preprint arXiv:1708.02596, 2017. · arXiv
- Ng, A. Y., Harada, D., and Russell, S. J. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML, 1999.
- Shi, J.-C., Yu, Y., Da, Q., Chen, S.-Y., and Zeng, A.-X. Virtual-taobao: Virtualizing real-world online retail en- vironment for reinforcement learning. In Thirty-Third AAAI Conference on Artificial Intelligence, 2019.
- Torabi, F., Warnell, G., and Stone, P. Behavioral cloning from observation. In IJCAI, 2018.
- Watkins, C. J. C. H. Learning from Delayed Rewards. PhD thesis, King's College, Oxford, May 1989. (To be reprinted by MIT Press.).
- Yang, S.-H., Long, B., Smola, A. J., Zha, H., and Zheng, Z. Collaborative competitive filtering: learning recom- mender using context of user choice. In Proceedings of the 34th international ACM SIGIR conference on Re- search and development in Information Retrieval, pp. 295-304. ACM, 2011.
- Zhao, X., Zhang, L., Ding, Z., Yin, D., Zhao, Y., and Tang, J. Deep reinforcement learning for list-wise recommen- dations. CoRR, 2018b.
Cited by
Sorted by citations
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
Self-citationReinforcing Pretrained Models for Generating Attractive Text Advertisements
Wang, X., Gu, X., Cao, J., Zhao, Z., Yan, Y., Middha, B., & Xie, X. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467105 · Cited by 20
Liu, J., Hao, W., Cheng, K., & Jin, D. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110319 · Cited by 5
Metadata
- Authors
- Chen, X., Li, S., Li, H., Jiang, S., Qi, Y., & Song, L.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- arXiv
- 1812.10613
- References
- 1
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative Adversarial User Model for Reinforcement Learning Based Recommendation System. https://app.cochat.ai/papers/10.48550/arxiv.1812.10613
- The paper
- Chen, X., Li, S., Li, H., Jiang, S., Qi, Y., & Song, L. (2018). Generative Adversarial User Model for Reinforcement Learning Based Recommendation System. arXiv. https://doi.org/10.48550/arxiv.1812.10613
