Leveraging Large Language Models for Sequential Recommendation
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Sequential recommendation problems have received increasing attention in research during the past few years, leading to the inception of a large variety of algorithmic approaches. In this work, we explore how large language models (LLMs), which are nowadays introducing disruptive effects in many AI-based applications, can be used to build or improve sequential recommendation approaches. Specifically, we devise and evaluate three approaches to leverage the power of LLMs in different ways. Our results from experiments on two datasets show that initializing the state-of-the-art sequential recommendation model BERT4Rec with embeddings obtained from an LLM improves NDCG by 15-20% compared to the vanilla BERT4Rec model. Furthermore, we find that a simple approach that leverages LLM embeddings for producing recommendations, can provide competitive performance by highlighting semantically related items. We publicly share the code and data of our experiments to ensure reproducibility.1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“LLMs as recommendation models can provide recommendations given a) only a task specification (zero-shot), b) a few examples given inline to the prompt of a task (few-shot), or c) after fine-tuning the model’s weights for a task given a set of training examples [4].”
MethodGroupLens: an open architecture for collaborative filtering of netnews
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., & Riedl, J. (1994) · Proceedings of the 1994 ACM conference on Computer supported cooperative work - CSCW '94 · doi:10.1145/192844.192905 · Cited by 5,058
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357895 · Cited by 2,558
Geng, S., Liu, S., Fu, Z., Ge, Y., & Zhang, Y. (2022) · Proceedings of the 16th ACM Conference on Recommender Systems · doi:10.1145/3523227.3546767 · Cited by 556
A Survey on Large Language Models for Recommendation
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.19860 · Cited by 515
A survey on large language models for recommendation
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01291-2 · Cited by 515
Bao, K., Zhang, J., Zhang, Y., Wang, W., Feng, F., & He, X. (2023) · Proceedings of the 17th ACM Conference on Recommender Systems · doi:10.1145/3604915.3608857 · Cited by 469 · cited 1×
“According to recent surveys [27, 40], LLMs are mainly utilized for recommendation problems in two ways: by providing embeddings that can be used to initialize existing recommendation models [29, 39, 43], and by producing recommendations leveraging their inherent knowledge encoding [2, 13, 22].”
BackgroundLarge Language Models are Zero-Shot Rankers for Recommender Systems
Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., & Zhao, W. X. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-56060-6_24 · Cited by 327
Towards Universal Sequence Representation Learning for Recommender Systems
Hou, Y., Mu, S., Zhao, W. X., Li, Y., Ding, B., & Wen, J.-R. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539381 · Cited by 313 · cited 1×
“In fact, LLMs show early indications of adaptability to different recommendation domains with modest fine-tuning [15, 16].”
BackgroundWhere to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited
Yuan, Z., Yuan, F., Song, Y., Li, Y., Fu, J., Yang, F., Pan, Y., & Ni, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.13835 · Cited by 267
How Can Recommender Systems Benefit from Large Language Models: A Survey
Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3678004 · Cited by 242 · cited 2×
“According to recent surveys [27, 40], LLMs are mainly utilized for recommendation problems in two ways: by providing embeddings that can be used to initialize existing recommendation models [29, 39, 43], and by producing recommendations leveraging their inherent knowledge encoding [2, 13, 22].”
BackgroundWhere to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited
Yuan, Z., Yuan, F., Song, Y., Li, Y., Fu, J., Yang, F., Pan, Y., & Ni, Y. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591932 · Cited by 200 · cited 3×
“The recent approaches presented in [15] and [42] differ from our work in particular in terms of the goals they pursue.”
BackgroundMethodLearning Vector-Quantized Item Representation for Transferable Sequential Recommenders
Hou, Y., He, Z., McAuley, J., & Zhao, W. X. (2023) · Proceedings of the ACM Web Conference 2023 · doi:10.1145/3543507.3583434 · Cited by 172 · cited 4×
“The recent approaches presented in [15] and [42] differ from our work in particular in terms of the goals they pursue.”
BackgroundMethodInfluentialIs ChatGPT a Good Recommender? A Preliminary Study
Liu, J., Liu, C., Zhou, P., Lv, R., Zhou, K., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.10149 · Cited by 60
Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction
Kang, W.-C., Ni, J., Mehta, N., Sathiamoorthy, M., Hong, L., Chi, E., & Cheng, D. Z. (2023) · arXiv · doi:10.48550/arxiv.2305.06474 · Cited by 28
Ding, H., Ma, Y., Deoras, A., Wang, Y., & Wang, H. (2021) · arXiv · doi:10.48550/arxiv.2105.08318 · Cited by 19
Cited by
A survey on large language models for recommendation
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01291-2 · Cited by 515
A Survey on Large Language Models for Recommendation
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.19860 · Cited by 515
How Can Recommender Systems Benefit from Large Language Models: A Survey
Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3678004 · Cited by 242
“Therefore, earlier works usually tend to either incorporate these small-scale language models as simple textual feature encoders, or as scoring/ranking functions finetuned to fit the data distribution from recommender systems.”
Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation
Zheng, B., Hou, Y., Lu, H., Chen, Y., Zhao, W. X., Chen, M., & Wen, J.-R. (2024) · 2024 IEEE 40th International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00118 · Cited by 143
“In this paper, we aim to combine LLMs and recommendation tasks in a more effective way, which is reached through the proposed new item indexing and alignment tuning methods.”
Method
Metadata
- Authors
- Harte, J., Zorgdrager, W., Louridas, P., Katsifodimos, A., Jannach, D., & Fragkoulis, M.
- Venue
- Proceedings of the 17th ACM Conference on Recommender Systems
- Publisher
- ACM
- Published
- 2023-09-14
- Pages
- 1096-1102
- OpenAlex
- W4386729952
- Cited by
- 128
- References
- 18
- FWCI
- 34.11
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.