Personalized Negative Reservoir for Incremental Learning in Recommender Systems
Open accessAbstract
Recommender systems have become an integral part of online platforms. Every day the volume of training data is expanding and the number of user interactions is constantly increasing. The exploration of larger and more expressive models has become a necessary pursuit to improve user experience. However, this progression carries with it an increased computational burden. In commercial settings, once a recommendation system model has been trained and deployed it typically needs to be updated frequently as new client data arrive. Cumulatively, the mounting volume of data is guaranteed to eventually make full batch retraining of the model from scratch computationally infeasible. Naively fine-tuning solely on the new data runs into the well-documented problem of catastrophic forgetting. Despite the fact that negative sampling is a crucial part of training with implicit feedback, no specialized technique exists that is tailored to the incremental learning framework. In this work, we propose a personalized negative reservoir strategy, which is used to obtain negative samples for the standard triplet loss of graph-based recommendation systems. Our technique balances alleviation of forgetting with plasticity by encouraging the model to remember stable user preferences and selectively forget when user interests change. We derive the mathematical formulation of a negative sampler to populate and update the reservoir. We integrate our design in three SOTA and commonly used incremental recommendation models. We show that these concrete realizations of our negative reservoir framework achieve state-of-the-art results for standard benchmarks using multiple top-k evaluation metrics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Storage costs can be further optimized by selectively retaining only a subset of training samples, such as through reservoir sampling.
“Storage costs can be further optimized by selectively retaining only a subset of training samples, such as through reservoir sampling technique (Valkanas et al., 2024), where new data is added over time while older data is removed.”
From Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation (2025) · cites this paper as Valkanas et al., 2024
Section 6.2 (page unavailable) describes reservoir replay methods that retain a subset of historical interactions and train with new data; Section 2.1 likewise describes replaying a reservoir of old data alongside new data. The paper does not establish that this reduces storage costs or that older stored data are removed as new data arrive. Its proposed reservoir is specifically for negative samples, not a general strategy for retaining training samples. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1× · see citing works
“where L BPR data batch and L 𝐾𝐷 teacher and student models, M 𝑇 and M 𝑆 KD incremental learning technique, L 𝐾𝐷”
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401063 · Cited by 5,083 · see citing works
Self-citationBPR: Bayesian Personalized Ranking from Implicit FeedbackPreprint
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 4,604 · cited 1× · see citing works
“where L BPR data batch and L 𝐾𝐷 teacher and student models, M 𝑇 and M 𝑆 KD incremental learning technique, L 𝐾𝐷”
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
Neural Graph Collaborative Filtering
Wang, X., He, X., Wang, M., Feng, F., & Chua, T.-S. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331267 · Cited by 3,823 · see citing works
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219890 · Cited by 3,733 · cited 2× · see citing works
“𝑖 ∈ I according to: where and I [ 𝑖 ∈Q] ( 𝑖 ) is an indicator function that is one when item 𝑖 belongs to the set Q of top negative items for user 𝑢 at ( 𝑁 𝑢, 1 , 𝑁 𝑢, |I| ) from (7) is a draw from the multinomial with”
Wide & Deep Learning for Recommender Systems
Cheng, H.-T., Koç, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G. S., Chai, W., Ispir, M., et al. (2016) · Workshop on Deep Learning for Recommender Systems (DLRS) · doi:10.1145/2988450.2988454 · Cited by 3,547 · see citing works
Deep Neural Networks for YouTube Recommendations
Covington, P., Adams, J., & Sargin, E. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959190 · Cited by 3,539 · see citing works
Self-Attentive Sequential Recommendation
Kang, W.-C., & McAuley, J. (2018) · Proceedings · doi:10.1109/icdm.2018.00035 · Cited by 3,229 · see citing works
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/239 · Cited by 2,615 · see citing works
End-to-End Incremental Learning
Castro, F. M., Marín-Jiménez, M. J., Guil, N., Schmid, C., & Alahari, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_15 · Cited by 1,184 · see citing works
Recurrent Neural Networks with Top-k Gains for Session-based Recommendations
Hidasi, B., & Karatzoglou, A. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3271761 · Cited by 969 · see citing works
Hierarchical Graph Representation Learning with Differentiable PoolingPreprint
Ying, R., You, J., Morris, C., Ren, X., Hamilton, W. L., & Leskovec, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08804 · Cited by 798 · see citing works
Structural Deep Clustering Network
Bo, D., Wang, X., Shi, C., et al. (2020) · The Web Conference · doi:10.1145/3366423.3380214 · Cited by 735
Attributed Graph Clustering: A Deep Attentional Embedding Approach
Wang, C., Pan, S., Hu, R., Long, G., Jiang, J. B., & Zhang, C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/509 · Cited by 619 · see citing works
Wang, J., Yu, L., Zhang, W., Gong, Y., Xu, Y., Wang, B., Zhang, P., & Zhang, D. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080786 · Cited by 569 · see citing works
Incremental Learning of Object Detectors without Catastrophic Forgetting
Shmelkov, K., Schmid, C., & Alahari, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.368 · Cited by 559 · see citing works
GDumb: A Simple Approach that Questions Our Progress in Continual Learning
Prabhu, A., Torr, P. H. S., & Dokania, P. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_31 · Cited by 497 · see citing works
Hoare, C. (1961) · Communications of the ACM · doi:10.1145/366622.366647 · Cited by 431
Improving pairwise learning for item recommendation from implicit feedback
Rendle, S., & Freudenthaler, C. (2014) · Web Search and Data Mining · doi:10.1145/2556195.2556248 · Cited by 423 · see citing works
Large scale image annotation: learning to rank with joint word-image embeddings
Weston, J., Samy Bengio, A., & Usunier, N. (2010) · Machine Learning · doi:10.1007/s10994-010-5198-3 · Cited by 398 · see citing works
Optimizing top-n collaborative filtering via dynamic negative item sampling
Zhang, W.-N., Chen, T.-Q., Wang, J., et al. (2013) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2484028.2484126 · Cited by 316
KBGAN: Adversarial Learning for Knowledge Graph Embeddings
Cai, L., & Wang, W. Y. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1133 · Cited by 289 · see citing works
Distilling Knowledge From Graph Convolutional Networks
Yang, Y., Qiu, J., Song, M., Tao, D., & Wang, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00710 · Cited by 269 · see citing works
Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer
Fan, Z., Liu, Z., Zhang, J., et al. (2021) · International Conference on Information and Knowledge Management · doi:10.1145/3459637.3482242 · Cited by 253
Graph Learning based Recommender Systems: A Review
Wang, S., Hu, L., Wang, Y., He, X., Sheng, Q. Z., Orgun, M. A., Cao, L., Ricci⋆, F., & Yu, P. S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/630 · Cited by 229 · cited 1× · see citing works
“…of 𝑄 after being transformed Kull-back–Leibler (KL) divergence [19] between 𝑃 and 𝑄 , we encourage In practice we first initialize 𝝁 1 , 𝝁 2 , . . . , 𝝁 𝐾 are updated via back-propagation based on the gradients of L KL 𝑝 𝑡𝑖 by applying a softmax function with temperature 𝜏 ∈ R + :”
Self-citationWord2vec applied to recommendation: hyperparameters matter
Caselles-Dupré, H., Lesaint, F., & Royo-Letelier, J. (2018) · ACM Conference on Recommender Systems · doi:10.1145/3240323.3240377 · Cited by 170
Multi-graph Convolution Collaborative Filtering
Sun, J., Zhang, Y., Ma, C., Coates, M., Guo, H., Tang, R., & He, X. (2019) · Proceedings · doi:10.1109/icdm.2019.00165 · Cited by 157 · see citing works
Self-citationStreaming Graph Neural Networks via Continual Learning
Wang, J.-S., Song, G., Wu, Y., & Wang, L. (2020) · International Conference on Information and Knowledge Management · doi:10.1145/3340531.3411963 · Cited by 154 · see citing works
NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding
Zhang, Y., Yao, Q., Shao, Y., & Chen, L. (2019) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2019.00061 · Cited by 145 · see citing works
Self-citationIncorporating GAN for Negative Sampling in Knowledge Representation Learning
Wang, P., Li, S., & Pan, R. (2018) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11536 · Cited by 136
Xu, Y., Zhang, Y., Guo, W., Guo, H., Tang, R., & Coates, M. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412754 · Cited by 87 · cited 2× · see citing works
“…of 𝑄 after being transformed Kull-back–Leibler (KL) divergence [19] between 𝑃 and 𝑄 , we encourage In practice we first initialize 𝝁 1 , 𝝁 2 , . . . , 𝝁 𝐾 are updated via back-propagation based on the gradients of L KL 𝑝 𝑡𝑖 by applying a softmax function with temperature 𝜏 ∈ R + :”
Self-citationImproving Latent Factor Models via Personalized Feature Projection for One Class Recommendation
Zhao, T., McAuley, J., & King, I. (2015) · International Conference on Information and Knowledge Management · doi:10.1145/2806416.2806511 · Cited by 83
A Survey on User Behavior Modeling in Recommender Systems
He, Z.-C., Liu, W.-W., Guo, W., et al. (2023) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2302.11087 · Cited by 70
Self-citationMotion Inbetweening via Deep Δ-Interpolator
Oreshkin, B. N., Valkanas, A., Harvey, F. G., et al. (2022) · arXiv.org · doi:10.48550/arxiv.2201.06701 · Cited by 32
Self-citationWang, Y., Zhang, Y., & Coates, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482117 · Cited by 27 · see citing works
Self-citationStructure Aware Experience Replay for Incremental Learning in Graph-based Recommender Systems
Ahrabian, K., Xu, Y., Zhang, Y., Wu, J., Wang, Y., & Coates, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482193 · Cited by 25 · cited 1× · see citing works
“…of 𝑄 after being transformed Kull-back–Leibler (KL) divergence [19] between 𝑃 and 𝑄 , we encourage In practice we first initialize 𝝁 1 , 𝝁 2 , . . . , 𝝁 𝐾 are updated via back-propagation based on the gradients of L KL 𝑝 𝑡𝑖 by applying a softmax function with temperature 𝜏 ∈ R + :”
Self-citationStructure Aware Incremental Learning with Personalized Imitation Weights for Recommender Systems
Wang, Y., Zhang, Y., Valkanas, A., et al. (2023) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2305.01204 · Cited by 24
Self-citationAdaptive Online Hyper-Parameters Tuning for Ad Event-Prediction Models
Aharon, M., Kagian, A., & Somekh, O. (2017) · The Web Conference · doi:10.1145/3041021.3054184 · Cited by 14
Dynamic Online Ensembles of Basis Expansions
Waxman, D., & Djuri'c, P. M. (2024) · Trans. Mach. Learn. Res. · doi:10.48550/arxiv.2405.01365 · Cited by 9
Negative Sampling with Adaptive Denoising Mixup for Knowledge Graph Embedding
Chen, X.-G., Zhang, W., Yao, Z., et al. (2023) · International Workshop on the Semantic Web · doi:10.48550/arxiv.2310.09781 · Cited by 9
MODL: Multilearner Online Deep Learning
Valkanas, A., Oreshkin, B. N., & Coates, M. (2024) · International Conference on Artificial Intelligence and Statistics · doi:10.48550/arxiv.2405.18281 · Cited by 2
Self-citationA Biased Sampling Method for Imbalanced Personalized Ranking
Yu, L., Pei, S., Zhu, F., et al. (2022) · International Conference on Information and Knowledge Management · doi:10.1145/3511808.3557218 · Cited by 2
Flagged first, then by citations
10 of the 56 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proc. ACM Int. Conf. Research and Development in Info. Retrieval (SIGIR), pp. 639-648, Online, July 2020a.
- Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. pp. 639-648, Online, Jul. 2020b. Published in Transactions on Machine Learning Research (02/2025)
- Rianne van den Berg, Thomas N. Kipf, and Max Welling. Graph convolutional matrix completion. In ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining -Deep Learning Day Workshop, London, United Kingdom, Aug. 2018.
- Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proc. Int. ACM Conf. Research and Dev. Info. Retrieval (SIGIR), pp. 515-524, Tokyo, Japan, Aug. 2017.
- Yishi Xu, Yingxue Zhang, Wei Guo, Huifeng Guo, Ruiming Tang, and Mark Coates. Graphsail: Graph struc- ture aware incremental learning for recommender systems. In Proc. ACM Int. Conf. Info. & Knowledge Management (CIKM), pp. 2861-2868, Online, Oct. 2020.
- Naimeng Yao, Qing Liu, Xiang Li, Yi Yang, and Quan Bai. Entity similarity-based negative sampling for knowledge graph embedding. In Pacific Rim Int. Conf. Artificial Intell., pp. 73-87. Springer, 2022.
Cited by
Sorted by citations
Fazulianov, D. V., & Guseva, A. I. (2024) · Современные наукоемкие технологии (Modern High Technologies) · doi:10.17513/snt.40009 · Cited by 1 · see citing works
Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation
Wang, C., Zhang, Y., Wang, Z., Shi, T., Bao, K., Feng, F., & Chua, T.-S. (2025) · arXiv · doi:10.48550/arxiv.2509.11524 · Cited by 2 · see citing works
Self-citation
- Authors
- Valkanas, A., Wang, Y., Zhang, Y., & Coates, M.
- Affiliations
- McGill University (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2403.03993
- OpenAlex
- W4392619043
- Semantic Scholar
- 8c659df0ec487a5e4f42d1f46ea93227a47e5fce
- Cited by
- 1
- References
- 56
- Citations by year
- 2024: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- incremental learning, recommender systems, catastrophic forgetting, negative sampling, triplet loss, graph-based recommendation, implicit feedback, online model updating
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
