CoChat

When large language models meet personalization: perspectives of challenges and opportunities

Open access
Chen, Jin, Liu, Zheng, Huang, Xu, Wu, Chenwang, Liu, Qi, Jiang, Gangwei, Pu, Yuanhao, Lei, Yuxuan, Chen, Xiaolong, Wang, Xingmei, Zheng, Kai, Lian, Defu, Chen, Enhong · 2024 · World Wide Web

The DOI matches its Crossref record; 329 of the 329 references it cites were found in a registry or an index; 90 of the 219 sources its claims cite were checked for support.

Abstract

Perspective paper arguing that LLMs change personalization from passive recommendation to active, tool-using personalized assistance.

Overview

CoChat Summary

TL;DR

This perspective/survey paper examines how large language models can be integrated into personalization systems, shifting from passive recommendation toward active, tool-augmented assistance. It maps LLM roles across personalization and outlines open challenges in deployment, model capacity, and evaluation.

Key findings

  • The paper organizes LLM use for personalization into roles including knowledge base, content interpreter, explainer, common system reasoner, conversational agent, tool-learning, and personalized content creator, arguing for a progression from low-level capability use to complex tool integration.
  • Zero-shot and few-shot LLM recommenders show some capability, with few-shot learners performing better than zero-shot, but the paper reports a substantial gap compared with traditional recommenders and finetuned LLMs such as P5 and M6-Rec.
  • In the knowledge-base role, LLMs can support knowledge graph completion and construction, but hallucination/ambiguity remains a challenge; on the KoLA benchmark, top-ranked GPT-4 achieves 0.012 Precision and 0.013 Recall on Named Entity Recognition, versus 0.712 Precision and 0.706 Recall for the task-specific model PL-Marker.
  • Open challenges include scaling computational resources, achieving efficient response times, laborious data collection, long-text modeling (e.g., ChatGPT’s 4096-token context window), interpretability/explainability, evaluation metrics, and trade-offs among helpfulness, honesty, and harmlessness.

Methods

  • This is a perspective/survey paper; the Data Availability statement says no datasets were generated or analysed.
  • The main analysis is a narrative synthesis of personalization techniques and LLM capabilities, with tables summarizing datasets, tasks, and approaches (e.g., LLMs as knowledge base, content interpreter, zero/few-shot learners, and tool-learning approaches).
  • No experimental design, statistical analysis, or primary empirical evaluation is reported.

Limitations

  • The paper states open challenges: deployment difficulties (computational resources, response time, data collection), LLM capacities (long text modeling, interpretability/explainability), and evaluation (metrics, helpfulness/honesty/harmlessness trade-off).
  • It notes that LLMs are black boxes, explanations may be insincere or inconsistent with recommendations, and training-data biases/errors can affect recommendations.
  • Evident from the text: as a perspective/survey, it does not provide primary empirical validation, and the Data Availability statement confirms no datasets were generated or analysed.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11280-024-01276-1

Cites

329 of 329 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
From the PDF
Image 2
From the PDF
Image 3
From the PDF
Figure 4 An example of zero/few-shot learning for direct recommenders
Figure 4 An example of zero/few-shot learning for direct recommendersFrom the PDF
Figure 5 Important approaches in conversational agent
Figure 5 Important approaches in conversational agentFrom the PDF
Figure 6 Advantages of content creator with LLMs
Figure 6 Advantages of content creator with LLMsFrom the PDF

Verification

Claims the paper makes

59 Valid · 50 Not valid · 48 Not checked · 105 Own results and interpretations

44 more claims · 10 with their cited sources checked

Create a free account to see all 262 claims and to verify the 82 references still open.

See all claims — free account
  • Probably not a substantial claim. 179
  • It may state two separate claims as one. 91

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Items 201–329 of 329

Research works printed but not linked: 17

  1. Bengio, Y., Ducharme, R., Vincent, P.: A neural probabilistic language model. Adv. Neural. Inf. Process. Syst. 13 (2000)
  2. Kenton, J.D.M.-W.C., Toutanova, L.K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)
  3. Yu, X., Ren, X., Gu, Q., Sun, Y., Han, J.: Collaborative filtering with entity similarity regularization in heterogeneous information networks. IJCAI HINA 27 (2013)
  4. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural. Inf. Process. Syst. 26 (2013)
  5. Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. In: ICML. JMLR Workshop and Conference Proceedings, vol. 32, pp. 1188–1196. JMLR.org (2014)
  6. Kumar, V., Khattar, D., Gupta, S., Gupta, M., Varma, V.: Deep neural architecture for news recommendation. In: CLEF (Working Notes). CEUR Workshop Proceedings, vol. 1866. CEUR-WS.org (2017)
  7. Liu, Q., Zhu, J., Dai, Q., Wu, X.: Boosting deep ctr prediction with a plug-and-play pre-trainer for news recommendation. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2823–2833 (2022)
  8. Muhamed, A., Keivanloo, I., Perera, S., Mracek, J., Xu, Y., Cui, Q., Rajagopalan, S., Zeng, B., Chilimbi, T.: Ctr-bert: cost-effective knowledge distillation for billion-parameter teacher models. In: NeurIPS Efficient Natural Language and Speech Processing Workshop (2021)
  9. Bills, S., Cammarata, N., Mossing, D., Tillman, H., Gao, L., Goh, G., Sutskever, I., Leike, J., Wu, J., Saunders, W.: Language models can explain neurons in language models (2023)
  10. Xie, S.M., Raghunathan, A., Liang, P., Ma, T.: An explanation of in-context learning as implicit bayesian inference. In: International Conference on Learning Representations
  11. Qiu, X., Huang, X.: Convolutional neural tensor network architecture for community-based question answering. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)
  12. Zhou, H., Huang, M., Zhu, X.: Context-aware natural language generation for spoken dialogue systems. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2032–2041 (2016)
  13. Li, J., Li, D., Xiong, C., Hoi, S.: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: International Conference on Machine Learning, pp. 12888–12900 (2022). PMLR
  14. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Adv. Neural. Inf. Process. Syst. 27 (2014)
  15. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763 (2021). PMLR
  16. Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: International Conference on Machine Learning, pp. 8821–8831 (2021). PMLR
  17. Hu, E.J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: low-rank adaptation of large language models. In: International Conference on Learning Representations (2021)

Cited by

See all 368

Metadata

Authors
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., Zheng, K., Lian, D., & Chen, E.
Affiliations
  • Beijing Academy of Artificial Intelligence (CN)
  • University of Science and Technology of China (CN)
  • University of Electronic Science and Technology of China (CN)
Corresponding author
  • Liu, Zheng
  • Huang, Xu
  • Wu, Chenwang
  • Liu, Qi
  • Jiang, Gangwei
  • Pu, Yuanhao
  • Lei, Yuxuan
  • Chen, Xiaolong
  • Wang, Xingmei
  • Zheng, Kai
  • Lian, Defu
  • Chen, Enhong
Venue
World Wide Web
Published
2024-06-28
Volume
27
Issue
4
ISSN
  • 1386-145X (ISSN-L)
  • 1573-1413
OpenAlex
W4400118952
Cited by
367
References
236
Citations by year
2026: 126 · 2025: 194 · 2024: 43
FWCI
135.88
Type
journal article
Language
en
Open access
Licence
Locations
  • World Wide Web (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
Hong Kong University of Science and Technology
Topics
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, personalization, personalized services, human-AI interaction, tool calling, function calling, recommender systems, search engines

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.