The DOI matches its Crossref record; 25 of the 27 references it cites were found in a registry or an index.
Conversational Recommender System
Open accessAbstract
A personalized conversational sales agent could have much commercial potential. E-commerce companies such as Amazon, eBay, JD, Alibaba etc. are piloting such kind of agents with their users. However, the research on this topic is very limited and existing solutions are either based on single round adhoc search engine or traditional multi round dialog system. They usually only utilize user inputs in the current session, ignoring users' long term preferences. On the other hand, it is well known that sales conversion rate can be greatly improved based on recommender systems, which learn user preferences based on past purchasing behavior and optimize business oriented metrics such as conversion rate or expected revenue. In this work, we propose to integrate research in dialog systems and recommender systems into a novel and unified deep reinforcement learning framework to build a personalized conversational recommendation agent that optimizes a per session based utility function. In particular, we propose to represent a user conversation history as a semi-structured user query with facet-value pairs. This query is generated and updated by belief tracker that analyzes natural language utterances of user at each step. We propose a set of machine actions tailored for recommendation agents and train a deep policy network to decide which action (i.e. asking for the value of a facet or making a recommendation) the agent should take at each step. We train a personalized recommendation model that uses both the user's past ratings and user query collected in the current conversational session when making rating predictions and generating recommendations. Such a conversational system often tries to collect user preferences by asking questions. Once enough user preference is collected, it makes personalized recommendations to the user. We perform both simulation experiments and real online user studies to demonstrate the effectiveness of the proposed framework.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We found that existing NLP datasets do not cover the wide breadth of possible conversational failure cases due to their emphasis on correct data for training.”
From A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures (2023) · cites this paper as [57]
In the citing paper
“Conversational Recommender Systems [24, 25] mark a remarkable stride forward in the realm of personalized assistance. By engaging users in interactive conversations, these systems delve deeper into their preferences and fine-tune their recommendations accordingly.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [24]
In the citing paper
“Based on the classification of dialogue systems, common approaches in conversational recommender systems can also be divided into two categories: attribute-based QA (questionanswering) and generative methods.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [24]
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R. M., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,232
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Probabilistic Matrix Factorization
Mnih, A., & Salakhutdinov, R. (2007) · neural information processing systems · Cited by 3,538
Rendle, S. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.127 · Cited by 3,189
Content-based Recommender Systems: State of the Art and Trends
Lops, P., de Gemmis, M., & Semeraro, G. M. (2010) · Springer eBooks · doi:10.1007/978-0-387-85820-3_3 · Cited by 1,941
An experimental comparison of click position-bias models
Craswell, N., Zoeter, O. R., Taylor, M. J. H., & Ramsey, B. (2008) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1341531.1341545 · Cited by 947
POMDP-Based Statistical Spoken Dialog Systems: A Review
Young, S. J., Gašić, M., Thomson, B., & Williams, J. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2225812 · Cited by 898
Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
Mesnil, G., Dauphin, Y., Yao, K., Bengio, Y., Deng, L., Hakkani-Tür, D. Z., He, X., Heck, L. P., Tür, G., Yu, D., et al. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2383614 · Cited by 484
Towards Conversational Recommender Systems
Christakopoulou, K., Radlinski, F., & Hofmann, K. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939746 · Cited by 445
The Dialog State Tracking Challenge
Williams, J. D., Raux, A., Ramachandran, D., & Black, A. W. (2013) · Cited by 315
Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access
Dhingra, B., Li, L. C., Li, X., Gao, J., Chen, Y.-N., Ahmed, F., & Deng, L. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1045 · Cited by 274
Real-Time Bidding by Reinforcement Learning in Display Advertising
Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Yu, Y., & Guo, D. (2017) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3018661.3018702 · Cited by 269
Personalized interactive faceted search
Koren, J., Zhang, Y., & Liu, X. (2008) · International World Wide Web Conference (WWW) · doi:10.1145/1367497.1367562 · Cited by 149
Self-citationReal-Time Bidding by Reinforcement Learning in Display Advertising
Cai, H., Ren, K., Zhag, W., Malialis, K., Wang, J., Yu, Y., & Guo, D. (2017) · UCL Discovery (University College London) · Cited by 115
Introduction to Information Retrieval
Kraft, D. H., & Colvin, E. (2017) · Synthesis lectures on information concepts, retrieval, and services · doi:10.1007/978-3-031-02307-1_1 · Cited by 85
Interactive retrieval based on faceted feedback
Zhang, L., & Zhang, Y. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835511 · Cited by 41
Self-citationDynamic Facet Ordering for Faceted Product Search Engines
Vandić, D., Aanen, S. S., Frăsincar, F., & Kaymak, U. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2652461 · Cited by 35
Wu, J. Y., Li, M., & Lee, C. (2015) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2015.2462712 · Cited by 23
Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue SystemsPreprint
Li, X., Chen, Y.-N., Li, L. C., Gao, J., & Çelikyılmaz, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07055 · Cited by 22
N-best error simulation for training spoken dialogue systems
Thomson, B., Gašić, M., Henderson, M., Tsiakoulis, P., & Young, S. J. (2012) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2012.6424194 · Cited by 19
Research works printed but not linked: 12
- Bordes Antoine, 2016, Learning end-to-end goal-oriented dialog. arXiv preprint arXiv:1605.07683 · arXiv
- Li Jiwei, 2015, A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055 · arXiv
- Lops Pasquale, 2011, Marco De Gemmis, and Giovanni Semeraro
- Narasimhan Karthik, 2016, Improving information extraction by acquiring external evidence with reinforcement learning. arXiv preprint arXiv:1603.07954 · arXiv
- Nogueira Rodrigo, 2017, Task-oriented query reformulation with reinforcement learning. arXiv preprint arXiv:1704.04572 · arXiv
- 10.5555/1046920.1088715 · DOI
- Silver David, 2016, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et almbox. .
- Silver David, 2017, et almbox
- Vinyals Oriol, 2015, A neural conversational model. arXiv preprint arXiv:1506.05869 · arXiv
- Wen Tsung-Hsien, 2016, A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562 · arXiv
- Williams Jason, 2013, Proceedings of the SIGDIAL 2013 Conference. 404--413
- Zhao Tiancheng, 2016, Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. arXiv preprint arXiv:1606.02560 · arXiv
Cited by
Sorted by citations
A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., Quan, Y., Chang, J., Jin, D., He, X., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3568022 · Cited by 896
Reinforcement Learning based Recommender Systems: A Survey
Afsar, M. M., Crump, T., & Far, B. H. (2022) · ACM Computing Surveys · doi:10.1145/3543846 · Cited by 637
When large language models meet personalization: perspectives of challenges and opportunities
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01276-1 · Cited by 367
“The attribute-based QA approach [24, 216–218] utilizes a pipeline method within the dialogue system.”
InfluentialAdvances and challenges in conversational recommender systems: A survey
Gao, C., Lei, W., He, X., de Rijke, M., & Chua, T. (2021) · AI Open · doi:10.1016/j.aiopen.2021.06.002 · Cited by 390
Asking Clarifying Questions in Open-Domain Information-Seeking Conversations
Aliannejadi, M., Zamani, H., Crestani, F., & Croft, W. B. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331265 · Cited by 356
Metadata
- Authors
- Sun, Y., & Zhang, Y.
- Affiliations
- University of California, Santa Cruz (US)
- ORCID
- Zhang, Yi
- Venue
- The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval
- Publisher
- ACM
- Published
- 2018-06-27
- Pages
- 235-244
- OpenAlex
- W2964112275
- Cited by
- 378
- References
- 27
- Citations by year
- 2026: 16 · 2025: 40 · 2024: 50 · 2023: 67 · 2022: 61 · 2021: 69 · 2020: 48 · 2019: 25 · 2018: 2
- FWCI
- 28.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- conversational recommender systems, personalized recommendation, deep reinforcement learning, dialogue systems, salespeople, e-commerce, long-term user preferences, belief tracking, rating prediction, conversion optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
