Making Recommender Systems Forget: Learning and Unlearning for Erasable Recommendation
Open accessAbstract
Privacy laws and regulations enforce data-driven systems, e.g., recommender systems, to erase the data that concern individuals. As machine learning models potentially memorize the training data, data erasure should also unlearn the data lineage in models, which raises increasing interest in the problem of Machine Unlearning (MU). However, existing MU methods cannot be directly applied into recommendation. The basic idea of most recommender systems is collaborative filtering, but existing MU methods ignore the collaborative information across users and items. In this paper, we propose a general erasable recommendation framework, namely LASER, which consists of Group module and SeqTrain module. Firstly, Group module partitions users into balanced groups based on their similarity of collaborative embedding learned via hypergraph. Then SeqTrain module trains the model sequentially on all groups with curriculum learning. Both theoretical analysis and experiments on two real-world datasets demonstrate that LASER can not only achieve efficient unlearning, but also outperform the state-of-the-art unlearning framework in terms of model utility.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Machine unlearning has played an essential role in many applications.
“Machine unlearning has played an essential role in many applications [33, 34].”
From Machine Unlearning: A Survey (2024) · cites this paper as [34]
The Introduction describes machine unlearning as important for erasing user data from recommendation models, and Related Work (§5) discusses methods for clustering and classification. The paper does not establish that machine unlearning has already played an essential role across many applications. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Neural Collaborative Filtering
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052569 · Cited by 7,066 · cited 2× · see citing works
“For conciseness, in this paper, we only consider rating data and choose two well-known CF models, i.e., Deep Matrix Factorization (DMF) [ Xue et al. , 2017 ] and Neural Matrix Factorization (NMF) [ He et al. , 2017 ] .”
MethodThe MovieLens Datasets: History and Context
Harper, F., & Konstan, J. (2016) · TIIS · doi:10.1145/2827872 · Cited by 5,938 · cited 1× · see citing works
“The ML and AM datasets are widely used to evaluate CF algorithms [Harper and Konstan, 2015; He and McAuley, 2016].”
MethodAn efficient k-means clustering algorithm: analysis and implementation
Kanungo, T., Mount, D. M., Netanyahu, N. S., Piatko, C. D., Silverman, R., & Wu, A. Y. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017616 · Cited by 5,660 · see citing works
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401063 · Cited by 5,083 · see citing works
Self-citationCollaborative Filtering for Implicit Feedback Datasets
Hu, Y., Koren, Y., & Volinsky, C. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.22 · Cited by 3,615 · see citing works
The EU General Data Protection Regulation (GDPR)
Russell, S. (2020) · doi:10.1093/oso/9780198826491.001.0001 · Cited by 2,103 · see citing works
Reynolds, D. A. (2009) · Encyclopedia of Biometrics · doi:10.1007/978-0-387-73003-5_196 · Cited by 2,062 · cited 1× · see citing works
“Method To achieve balanced grouping, we propose a general method that can be applied to most existing clustering algorithms, including but not limited to k -means clustering, label propagation algorithm [ Wang and Zhang, 2007 ] , and Gaussian mixture models [ Reynolds, 2009 ] .”
MethodCollaborative filtering recommender systems
Nilashi, M., Bagherifard, K., Ibrahim, O., et al. (2013) · Research Journal of Applied Sciences Engineering and Technology · doi:10.19026/rjaset.5.4644 · Cited by 1,890 · see citing works
He, R., & McAuley, J. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883037 · Cited by 1,718 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Higher-order organization of complex networks
Benson, A. R., Gleich, D. F., & Leskovec, J. (2016) · Science · doi:10.1126/science.aad9029 · Cited by 1,284 · see citing works
Rokach, L., & Maimon, O. (2005) · The Data Mining and Knowledge Discovery Handbook · doi:10.1007/978-1-4419-9863-7_370 · Cited by 1,125
Deep Matrix Factorization Models for Recommender Systems
Xue, H.-J., Dai, X., Zhang, J., Huang, S., & Chen, J. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/447 · Cited by 982 · see citing works
Church, K. (2016) · Natural Language Engineering · doi:10.1017/s1351324916000334 · Cited by 982 · cited 3× · see citing works
“…embedding: (i) using R to build the corresponding hypergraph; (ii) for each user, performing random walk to obtain its relations, which transforms the task to sequence embedding; (iii) applying the sequence embedding technique, e.g., Word2Vec [Church, 2017], to learn the collaborative embedding.”
MethodInfluentialLabel Propagation through Linear Neighborhoods
Wang, F., & Zhang, C.-S. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1145/1143844.1143968 · Cited by 913 · cited 1× · see citing works
“Method To achieve balanced grouping, we propose a general method that can be applied to most existing clustering algorithms, including but not limited to k -means clustering, label propagation algorithm [ Wang and Zhang, 2007 ] , and Gaussian mixture models [ Reynolds, 2009 ] .”
MethodBourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · see citing works
Collaborative Filtering beyond the User-Item Matrix
Shi, Y., Larson, M., & Hanjalić, A. (2014) · ACM Computing Surveys · doi:10.1145/2556270 · Cited by 805 · cited 1× · see citing works
“The reason is that the basic idea of most RSs is Collaborative Filtering (CF) [Shi et al., 2014], but these MU methods fail to consider the collaborative information across users and items.”
BackgroundA Survey on Curriculum Learning
Wang, X., Chen, Y., & Zhu, W. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3069908 · Cited by 692 · see citing works
An Entropy Weighting k-Means Algorithm for Subspace Clustering of High-Dimensional Sparse Data
Jing, L., Ng, M., & Huang, J. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.1048 · Cited by 644 · see citing works
Towards Making Systems Forget with Machine Unlearning
Cao, Y., & Yang, J. (2015) · 2015 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2015.35 · Cited by 622 · cited 1× · see citing works
“Cao and Yang [ 2015 ] firstly formed the task of MU and proposed a statistical query learning based unlearning framework.”
BackgroundNeural Collaborative Filtering vs. Matrix Factorization Revisited
Rendle, S., Krichene, W., Zhang, L., & Anderson, J. (2020) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3383313.3412488 · Cited by 511 · see citing works
He, X., Chen, T., Kan, M., & Chen, X. (2015) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2806416.2806504 · Cited by 465 · see citing works
Higher order learning with graphs
Agarwal, S., Branson, K., & Belongie, S. J. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143847 · Cited by 463 · cited 2×
“Hypergraph is a generalized structure for relation modeling, in which a hyper-edge connects two or more vertices.”
BackgroundIEEE Transactions on Knowledge and Data Engineering
(2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.1269660 · Cited by 301 · cited 1× · see citing works
“Method To achieve balanced grouping, we propose a general method that can be applied to most existing clustering algorithms, including but not limited to k-means clustering, label propagation algorithm [Wang and Zhang, 2007], and Gaussian mixture models [Reynolds, 2009].”
MethodOn The Power of Curriculum Learning in Training Deep NetworksPreprint
Hacohen, G., & Weinshall, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03626 · Cited by 286 · see citing works
Dual Channel Hypergraph Collaborative Filtering
Ji, S., Feng, Y.-F., Ji, R., et al. (2020) · Knowledge Discovery and Data Mining · doi:10.1145/3394486.3403253 · Cited by 255 · cited 2×
“Secondly, the original R or its corresponding bipartite graph cannot sufficiently represent the user-item collaboration, especially high-order relations [ Ji et al. , 2020 ] .”
BackgroundMethodData poisoning attacks on factorization-based collaborative filtering
Li, B., Wang, Y., Singh, A., & Vorobeychik, Y. (2016) · arXiv · doi:10.48550/arxiv.1608.08182 · Cited by 235 · see citing works
Alonso-Fernandez, F., & Fierrez, J. (2009) · Encyclopedia of Biometrics · doi:10.1007/978-0-387-73003-5 · Cited by 222 · cited 1× · see citing works
“Method To achieve balanced grouping, we propose a general method that can be applied to most existing clustering algorithms, including but not limited to k-means clustering, label propagation algorithm [Wang and Zhang, 2007], and Gaussian mixture models [Reynolds, 2009].”
MethodMultiple Incremental Decremental Learning of Support Vector Machines
Karasuyama, M., & Takeuchi, I. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2048039 · Cited by 176 · see citing works
Making AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175 · cited 1× · see citing works
“Ginart et al. [ 2019 ] proposed efficient unlearning algorithms for k -means clustering with bounds of unlearning time complexity.”
BackgroundA CAUCHY-SCHWARZ INEQUALITY FOR OPERATORS WITH APPLICATIONS
Bhatia, R., & Davis, C. (1995) · doi:10.1016/0024-3795(94)00344-d · Cited by 170 · cited 1×
“Based on Multinomial Theorem, we can rewrite (7) as: According to Cauchy-Schwarz Inequality [ Bhatia and Davis, 1995 ] , for n random variables x i , y i , we have: Setting y i = 1 for all i , we can get a lower bound of i x 2 i as: Taking it into (8), we have: We can easily compute that setting c…”
MethodRandom Walks on Hypergraphs with Edge-Dependent Vertex Weights
Chitra, U., & Raphael, B. J. (2019) · International Conference on Machine Learning · doi:10.48550/arxiv.1905.08287 · Cited by 134
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2022) · Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3548606.3559352 · Cited by 133 · see citing works
Machine unlearning: linear filtration for logit-based classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2022) · Machine Learning · doi:10.1007/s10994-022-06178-9 · Cited by 123 · see citing works
Chen, C., Sun, F., Zhang, M., & Ding, B. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3511997 · Cited by 110 · see citing works
Self-citationMachine Unlearning: Linear Filtration for Logit-based Classifiers
Baumhauer, T., Schöttle, P., & Zeppelzauer, M. (2020) · arXiv · doi:10.48550/arxiv.2002.02730 · Cited by 110 · see citing works
PUMA: Performance Unchanged Model Augmentation for Training Data Removal
Wu, G., Hashemi, M., & Srinivasa, C. (2022) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2203.00846 · Cited by 108
Remember What You Want to Forget: Algorithms for Machine Unlearning
Sekhari, A., Acharya, J., Kamath, G., & Suresh, A. T. (2021) · arXiv · doi:10.48550/arxiv.2103.03279 · Cited by 99 · cited 1× · see citing works
“, retrain unlearning and reverse unlearning, have been proposed [Bourtoule et al., 2021; Schelter et al., 2021; Sekhari et al., 2021].”
BackgroundARCANE: An Efficient Architecture for Exact Machine Unlearning
Yan, H., Li, X., Guo, Z., Li, H., Li, F., & Lin, X. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/556 · Cited by 97 · see citing works
HedgeCut: Maintaining Randomised Trees for Low-Latency Machine Unlearning
Schelter, S., Grafberger, S., & Dunning, T. (2021) · Proceedings of the 2021 International Conference on Management of Data · doi:10.1145/3448016.3457239 · Cited by 89 · cited 2× · see citing works
“To overcome the issue of inefficiency, two classes of MU methods, i.e., retrain unlearning and reverse unlearning, have been proposed [Bourtoule et al., 2021; Schelter et al., 2021; Sekhari et al., 2021].”
BackgroundMethodOn the Accuracy of Influence Functions for Measuring Group EffectsPreprint
Koh, P. W., Ang, K.-S., Teo, H. H. K., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13289 · Cited by 78 · see citing works
Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
(2020) · doi:10.1145/3394486 · Cited by 63 · cited 2× · see citing works
“Following [Ji et al., 2020], we set l = 4 in this paper.”
BackgroundMethodDeep Unlearning via Randomized Conditionally Independent Hessians
Mehta, R., Pal, S., Singh, V. P., & Ravi, S. N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01017 · Cited by 62 · see citing works
GIF: A General Graph Unlearning Strategy via Influence Function
Wu, J., Yang, Y., Qian, Y., Sui, Y., Wang, X., & He, X. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583521 · Cited by 58 · see citing works
Selective and Collaborative Influence Function for Efficient Recommendation Unlearning
Li, Y., Chen, C., Zheng, X.-L., et al. (2023) · Expert systems with applications · doi:10.48550/arxiv.2304.10199 · Cited by 50
Self-citationUnlearning Protected User Attributes in Recommendations with Adversarial Training
Ganhör, C., Penz, D., Rekabsaz, N., Lesota, O., & Schedl, M. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531820 · Cited by 47 · see citing works
Making Users Indistinguishable: Attribute-wise Unlearning in Recommender Systems
Li, Y., Chen, C., Zheng, X., Zhang, Y., Han, Z., Meng, D., & Wang, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612418 · Cited by 26 · see citing works
Self-citationRethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives
Xu, D., Ruan, C., Körpeoglu, E., et al. (2021) · International Conference on Machine Learning · doi:10.48550/arxiv.2110.12141 · Cited by 20
Graves, L., Nagisetty, V., & Ganesh, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17371 · Cited by 5 · see citing works
Flagged first, then by citations
6 of the 55 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Kenneth Ward Church. Word2vec. Natural Language Engineering, 23(1):155-162, 2017.
- He et al., 2015] Xiangnan He, Tao Chen, Min-Yen Kan, and Xiao Chen. Trirank: Review-aware explainable recom- mendation by modeling aspects. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM), pages 1661-1670, 2015.
- He et al., 2017] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collab- orative filtering. In Proceedings of the 26th International Conference on World Wide Web (WWW), pages 173-182, 2017.
- J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. Collaborative filtering recom- mender systems. In The adaptive web, pages 291-324. Springer, 2007.
Cited by
Sorted by citations
Xu, H., Zhu, T., Zhang, L., Zhou, W., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3603620 · Cited by 164 · see citing works
“Machine unlearning has played an essential role in many applications [33, 34].”
BackgroundA Review on Machine Unlearning
Zhang, H., Nakamura, T., Isohara, T., & Sakurai, K. (2023) · SN Computer Science · doi:10.1007/s42979-023-01767-4 · Cited by 81 · see citing works
“The data lineage management system can monitor any data changes in the machine learning model that occur at any point in time [56].”
BackgroundRecommendation Unlearning via Influence Function
Zhang, Y., Hu, Z., Bai, Y., Wu, J., Wang, Q., & Feng, F. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3701763 · Cited by 35 · see citing works
“Recently, recommendation unlearning receives certain attention [6, 24, 27, 50, 53].”
MethodInfluentialPost-Training Attribute Unlearning in Recommender Systems
Chen, C., Zhang, Y., Li, Y., Wang, J., Qi, L., Xu, X., Zheng, X., & Yin, J. (2024) · ACM Transactions on Information Systems · doi:10.1145/3701987 · Cited by 26 · see citing works
Self-citationDang, Y., Liu, Y., Yang, E., Guo, G., Jiang, L., Zhao, J., & Wang, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3706633 · Cited by 12 · see citing works
Intra- and Inter-group Optimal Transport for User-Oriented Fairness in Recommender Systems
Han, Z., Chen, C., Zheng, X., Li, M., Liu, W., Yao, B., Li, Y., & Yin, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i8.28689 · Cited by 8 · see citing works
“RS is a complex field involving frequent interactions between users and items (Su et al. 2023; Zheng et al. 2022b; Li et al. 2022, 2023).”
BackgroundSelf-citation
- Authors
- Li, Y., Zheng, X., Chen, C., & Liu, J.
- Affiliations
- Zhejiang University (CN)
- Corresponding author
- Zheng, Xiaolin
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Volume
- 283
- Pages
- 111124-111124
- ISSN
- 0950-7051 (ISSN-L)
- arXiv
- 2203.11491
- OpenAlex
- W4388069167
- Cited by
- 43
- References
- 55
- Citations by year
- 2026: 3 · 2025: 23 · 2024: 13 · 2023: 4
- FWCI
- 3.81
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Knowledge-Based Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- machine unlearning, recommender systems, data deletion, privacy, hypergraph embedding, data sharding, curriculum learning, early stopping, model utility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
