The DOI matches its DataCite record; the references it cites are not checked yet.
LaMP: When Large Language Models Meet Personalization
Open accessAbstract
This paper highlights the importance of personalization in large language models and introduces the LaMP benchmark -- a novel benchmark for training and evaluating language models for producing personalized outputs. LaMP offers a comprehensive evaluation framework with diverse language tasks and multiple entries for each user profile. It consists of seven personalized tasks, spanning three text classification and four text generation tasks. We additionally propose two retrieval augmentation approaches that retrieve personal items from each user profile for personalizing language model outputs. To this aim, we study various retrieval models, including term matching, semantic matching, and time-aware methods. Extensive experiments on LaMP for zero-shot and fine-tuned language models demonstrate the efficacy of the proposed retrieval augmentation approach and highlight the impact of personalization in various natural language tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“LaMP [29], PersonaLLM [30], and MemPrompt [31] explore personalization through prompting or fine-tuning.”
From From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression (2026) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 19
- Bruce W. Croft, Stephen Cronen-Townsend, and Victor Lavrenko. 2001. Relevance feedback and personal- ization: A language modeling perspective. In DE- LOS Workshop: Personalisation and Recommender Systems in Digital Libraries.
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In North American Chapter of the Association for Computational Linguistics.
- Susan T. Dumais. 2016. Personalized search: Potential and pitfalls. In Proceedings of the 25th ACM Interna- tional on Conference on Information and Knowledge Management, CIKM '16, page 689, New York, NY, USA. Association for Computing Machinery. · DOI
- Manish Gupta, Rui Li, Zhijun Yin, and Jiawei Han. 2010. Survey on social tagging techniques. SIGKDD Explor. Newsl.
- Xiaolei Huang, Lucie Flek, Franck Dernoncourt, Charles Welch, Silvio Amir, Ramit Sawhney, and Diyi Yang. 2022. Usernlp'22: 2022 international workshop on user-centered natural language process- ing. In Companion Proceedings of the Web Confer- ence 2022, WWW '22, page 1176-1177, New York, NY, USA. Association for Computing Machinery. Natural Language Processing and the 9th Interna- tional Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 188-197, Hong Kong, China. Association for Computational Lin- guistics. · DOI
- Oard, Douglas, Webber, William, Kirsch, David A., and Golitsynskiy, Sergey. 2015. Avocado research email collection. OpenAI. 2023. Gpt-4 technical report. · DOI
- Barbara Plank. 2022. The "problem" of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 10671-10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Joan Plepi, Béla Neuendorf, Lucie Flek, and Charles Welch. 2022. Unifying data perspectivism and per- sonalization: An application to social norms. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 7391- 7402, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Hongjin Qian, Xiaohe Li, Hanxun Zhong, Yu Guo, Yueyuan Ma, Yutao Zhu, Zhanliang Liu, Zhicheng Dou, and Ji-Rong Wen. 2021. Pchatbot: A large- scale dataset for personalized chatbot. In Proceed- ings of the 44th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, SIGIR '21, page 2470-2477, New York, NY, USA. Association for Computing Machinery. · DOI
- Omid Rafieian and Hema Yoganarasimhan. 2023. Ai and personalization. Artificial Intelligence in Market- ing, pages 77-102.
- Stephen Robertson, S. Walker, S. Jones, M. M. Hancock- Beaulieu, and M. Gatford. 1995. Okapi at trec-3. In Proceedings of the Third Text REtrieval Conference, TREC-3, pages 109-126.
- Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: Extraction and mining of academic social networks. In KDD'08, pages 990-998.
- Sebastian Vincent, Rowanne Sumner, Alice Dowek, Charlotte Blundell, Emily Preston, Chris Bayliss, Chris Oakley, and Carolina Scarton. 2023. Per- sonalised language modelling of screen characters using rich metadata annotations. arXiv preprint arXiv:2303.16618. · arXiv
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Aman- preet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understand- ing systems. In Advances in Neural Information Processing Systems, volume 32.
- Joern Wuebker, Patrick Simianer, and John DeNero. 2018. Compact personalized models for neural ma- chine translation. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing, pages 881-886, Brussels, Belgium. As- sociation for Computational Linguistics. · DOI
- Jiajing Xu, Andrew Zhai, and Charles Rosenberg. 2022. Rethinking personalized ranking at pinterest: An end- to-end approach. In Proceedings of the 16th ACM Conference on Recommender Systems, RecSys '22, page 502-505, New York, NY, USA. Association for Computing Machinery. · DOI
- Hansi Zeng, Surya Kallumadi, Zaid Alibadi, Rodrigo Nogueira, and Hamed Zamani. 2023. A personalized dense retrieval framework for unified information ac- cess. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '23, New York, NY, USA. Association for Computing Machinery.
- Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.
- Jian Zhu and David Jurgens. 2021. Idiosyncratic but not arbitrary: Learning idiolects in online registers reveals distinctive yet consistent individual styles. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.
Cited by
Sorted by citations
When large language models meet personalization: perspectives of challenges and opportunities
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01276-1 · Cited by 367
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
Large Language Models as Zero-Shot Conversational Recommenders
He, Z., Xie, Z., Jha, R., Steck, H., Liang, D., Feng, Y., Majumder, B. P., Kallus, N., & McAuley, J. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614949 · Cited by 201
Multimodal LLMs for Health Grounded in Individual-Specific Data
Belyaeva, A., Cosentino, J., Hormozdiari, F., Eswaran, K., Shetty, S., Corrado, G. S., Carroll, A. W., McLean, C. Y., & Furlotte, N. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47679-2_7 · Cited by 74
Wang, R., Gao, Z., Zhang, L., Yue, S., & Gao, Z. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100755 · Cited by 61
Metadata
- Authors
- Salemi, A., Mysore, S., Bendersky, M., & Zamani, H.
- Affiliations
- University of Massachusetts Amherst (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 7370-7392
- ISSN
- 0736-587X (ISSN-L)
- arXiv
- 2304.11406
- OpenAlex
- W4402671806
- Cited by
- 109
- References
- 0
- Citations by year
- 2026: 42 · 2025: 47 · 2024: 11 · 2023: 9
- FWCI
- 29.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2143434)
- Office of Naval Research (N000142212688)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- personalized language models, retrieval augmentation, user profiles, text classification, text generation, semantic matching, zero-shot learning, fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
