The DOI matches its Crossref record, with a small difference in title or year; 47 of the 48 references it cites were found in a registry or an index.
Popularity Bias is not Always Evil: Disentangling Benign and Harmful Bias for Recommendation
Open accessAbstract
Recommender system usually suffers from severepopularity bias— the collected interaction data usually exhibits quite imbalanced or even long-tailed distribution over items. Such skewed distribution may result from the users’conformityto the group, which deviates from reflecting users’ true preference. Existing efforts for tackling this issue mainly focus on completely eliminating popularity bias. However, we argue that not all popularity bias is evil. Popularity bias not only results from conformity but alsoitem quality, which is usually ignored by existing methods. Some items exhibit higher popularity as they have intrinsic better property. Blindly removing the popularity bias would lose such important signal, and further deteriorate model performance. To sufficiently exploit such important information for recommendation, it is essential to disentangle the benign popularity bias caused by item quality from the harmful popularity bias caused by conformity. Although important, it is quite challenging as we lack an explicit signal to differentiate the two factors of popularity bias. In this paper, we propose to leverage temporal information as the two factors exhibit quite different patterns along the time: item quality revealing item inherent property is stable and static while conformity that depends on items’ recent clicks is highly time-sensitive. Correspondingly, we further propose a novelTime-awareDisEntangled framework (TIDE), where a click is generated from three components namely the static item quality, the dynamic conformity effect, as well as the user-item matching score returned by any recommendation model. Lastly, we conduct interventional inference so that the recommendation can benefit from the benign popularity bias while circumvent the harmful one. Extensive experiments on four real-world datasets demonstrated the effectiveness of TIDE.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“An appropriate proportion of popularity bias in the system can improve performance and user satisfaction [183].”
From A Deep Dive into Fairness, Bias, Threats, and Privacy in Recommender Systems: Insights and Future Research (2024) · cites this paper as [183]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Moreover, popularity bias can act as a double-edged sword [182, 183].”
From A Deep Dive into Fairness, Bias, Threats, and Privacy in Recommender Systems: Insights and Future Research (2024) · cites this paper as [183]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Neural Collaborative Filtering
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052569 · Cited by 7,385 · cited 1×
“experiments, and it would be straightforward to replace it with more sophisticated models such as Factorization Machine [15], or Neural Network [5], [16].”
MethodSelf-citationBPR: Bayesian Personalized Ranking from Implicit FeedbackPreprint
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 6,168
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401063 · Cited by 5,300 · cited 4×
“Amazon-CDs is increased to 8,192 for speed and the number of convolutional layers is searched in {2, 3, 4} as advised by [5].”
BackgroundMethodInfluentialSelf-citationPearson Correlation Coefficient
Benesty, J., Chen, J., Huang, Y. A., & Cohen, I. (2009) · Springer topics in signal processing · doi:10.1007/978-3-642-00296-0_5 · Cited by 3,540
Self-citationThe long tail: why the future of business is selling less of more
(2007) · Choice Reviews Online · doi:10.5860/choice.44-2783 · Cited by 2,869
Deep Interest Network for Click-Through Rate Prediction
Zhou, G., Zhu, X., Song, C., Fan, Y., Zhu, H., Ma, X., Yan, Y., Jin, J., Li, H., & Gai, K. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219823 · Cited by 2,128 · cited 2×
“(3) ( U, I ) → M → Y : these links project user and item features ( e.g., IDs) into their matching scores m ui = f θ ( u, i ) . f θ ( u, i ) can be implemented by various recommendation models, such as MF [4], LightGCN [5], DIN [6], etc. Finally these three components are aggregated into a final prediction score for r…”
BackgroundFactorization Machines with libFM
Rendle, S. (2012) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2168752.2168771 · Cited by 1,345 · cited 1×
“Matrix Factorization (MF) has been selected as the main backbone recommendation model for experiments, and it would be straightforward to replace it with more sophisticated models such as Factorization Machine [15], or Neural Network [5], [16].”
MethodBias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
Self-citationDisentangled Graph Collaborative Filtering
Wang, X., Jin, H., Zhang, A., He, X., Xu, T., & Chua, T.-S. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401137 · Cited by 737
Self-citationUnbiased Learning-to-Rank with Biased Feedback
Joachims, T., Swaminathan, A., & Schnabel, T. (2017) · Proceedings of the Tenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3018661.3018699 · Cited by 531 · cited 3×
“MF-IPS [10], [11]: a classic strategy for eliminating popularity bias by re-weighting each instance according to item popularity.”
BackgroundInfluentialCounterfactual reasoning and learning systems: the example of computational advertising
Bottou, L., Peters, J., Quiñonero-Candela, J., Charles, D. X., Chickering, D. M., Portugaly, E., Ray, D., Simard, P., & Snelson, E. (2013) · Cited by 530
The long tail of recommender systems and how to leverage it
Park, Y., & Tuzhilin, A. (2008) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1454008.1454012 · Cited by 444 · cited 1×
“Worse still, recommendation model not only inherits the bias, but also amplifies bias, making the popular items dominate the top recommendations [13], [19], [20], [21], [22], [23].”
BackgroundCausal Intervention for Leveraging Popularity Bias in Recommendation
Zhang, Y., Feng, F., He, X., Wei, T., Song, C., Ling, G., & Zhang, Y. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462875 · Cited by 443 · cited 10×
“As recent work [2], here we simply use a uniform”
BackgroundMethodInfluentialSelf-citationModeling User Exposure in Recommendation
Liang, D., Charlin, L., McInerney, J., & Blei, D. M. (2016) · International World Wide Web Conference (WWW) · doi:10.1145/2872427.2883090 · Cited by 381 · cited 1×
“Besides popularity bias, recent works have studied other types of biases in recommendation including: Selection bias, which happens as users are free to choose which items to rate, so that the observed ratings are not a representative sample of all ratings [36], [37], [38], [39]; Exposure bias, which happens in implic…”
BackgroundDisentangling User Interest and Conformity for Recommendation with Causal Embedding
Zheng, Y., Gao, C., Li, X., He, X., Li, Y., & Jin, D. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449788 · Cited by 356
Self-citationRecommendations as Treatments: Debiasing Learning and EvaluationPreprint
Schnabel, T., Swaminathan, A., Singh, A., Chandak, N., & Joachims, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.05352 · Cited by 351
How algorithmic confounding in recommendation systems increases homogeneity and decreases utility
Chaney, A. J. B., Stewart, B., & Engelhardt, B. E. (2018) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3240323.3240370 · Cited by 345 · cited 1×
“It would decrease serendipity [25], [26], [27] and fairness [13], [28], [29], [30] of recommendation results, hurting user experience and causing customer churn.”
BackgroundModel-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
Wei, T., Feng, F., Chen, J., Wu, Z., Yi, J., & He, X. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467289 · Cited by 345 · cited 4×
“most successful and representative strategy [2], [7], [8].”
BackgroundMethodInfluentialSelf-citationCollaborative prediction and ranking with non-random missing data
Marlin, B. M., & Zemel, R. S. (2009) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1639714.1639717 · Cited by 319 · cited 1×
“Besides popularity bias, recent works have studied other types of biases in recommendation including: Selection bias, which happens as users are free to choose which items to rate, so that the observed ratings are not a representative sample of all ratings [36], [37], [38], [39]; Exposure bias, which happens in implic…”
BackgroundCausal embeddings for recommendation
Bonner, S., & Vasile, F. (2018) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3240323.3240360 · Cited by 275 · cited 2×
“[33] leverage regularization to distill knowledge from the uniform data for addressing popularity bias.”
BackgroundFrom Niches to Riches: The Anatomy of the Long Tail
Brynjolfsson, E., Hu, Y. J., & Smith, M. D. (2018) · Figshare · doi:10.1184/r1/6471422 · Cited by 250
Feedback Loop and Bias Amplification in Recommender Systems
Mansoury, M., Abdollahpouri, H., Pechenizkiy, M., Mobasher, B., & Burke, R. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412152 · Cited by 240 · cited 1×
“but also amplifies bias, making the popular items dominate the top recommendations [13], [19], [20], [21], [22], [23].”
BackgroundLearning Disentangled Representations for RecommendationPreprint
Ma, J., Zhou, C., Cui, P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.14238 · Cited by 197 · cited 1×
“[45] leverage Variational Auto-Encoder [46] to disentangle high-level concepts associated with user intentions as well as low-level factors (e.”
BackgroundThe Connection Between Popularity Bias, Calibration, and Fairness in Recommendation
Abdollahpouri, H., Mansoury, M., Burke, R., & Mobasher, B. (2020) · Fourteenth ACM Conference on Recommender Systems · doi:10.1145/3383313.3418487 · Cited by 183 · cited 1×
“It would decrease serendipity [25], [26], [27] and fairness [13], [28], [29], [30] of recommendation results, hurting user experience and causing customer churn.”
BackgroundUnderstanding Echo Chambers in E-commerce Recommender Systems
Ge, Y., Zhao, S., Zhou, H., Pei, C., Sun, F., Ou, W., & Zhang, Y. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401431 · Cited by 176
Celma, Ò., & Cano, P. (2008) · doi:10.1145/1722149.1722154 · Cited by 170 · cited 1×
“Worse still, recommendation model not only inherits the bias, but also amplifies bias, making the popular items dominate the top recommendations [13], [19], [20], [21], [22], [23].”
BackgroundOffline Evaluation to Make Decisions About PlaylistRecommendation Algorithms
Gruson, A., Chandar, P., Charbuillet, C., McInerney, J., Hansen, S., Tardieu, D., & Carterette, B. (2019) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3289600.3291027 · Cited by 161 · cited 1×
“classified into four types: (1) Inverse propensity scoring (IPS) [11], [31] is a classic strategy that directly adjust the data distribution with re-weighting each instance according”
BackgroundPopularity-Opportunity Bias in Collaborative Filtering
Zhu, Z., He, Y., Zhao, X., Zhang⋆, Y., Wang, J., & Caverlee, J. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441820 · Cited by 147 · cited 2×
“As PDA demonstrates superior performance over ranking-based methods [13], [14], we do not include these methods as baselines.”
MethodManaging Popularity Bias in Recommender Systems with Personalized Re-rankingPreprint
Abdollahpouri, H., Burke, R., & Mobasher, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07555 · Cited by 132 · cited 4×
“but also amplifies bias, making the popular items dominate the top recommendations [13], [19], [20], [21], [22], [23].”
BackgroundMethodInfluentialCañamares, R., & Castells, P. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210014 · Cited by 125 · cited 1×
“For example, some work [7], [18] argues that such skewed distribution may be caused by user conformity, deviating from reflecting”
BackgroundThe Unfairness of Popularity Bias in RecommendationPreprint
Abdollahpouri, H., Mansoury, M., Burke, R., & Mobasher, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13286 · Cited by 125 · cited 1×
“It would decrease serendipity [25], [26], [27] and fairness [13], [28], [29], [30] of recommendation results, hurting user experience and causing customer churn.”
BackgroundCorrecting for Selection Bias in Learning-to-rank Systems
Ovaisi, Z., Ahsan, R., Zhang, Y., Vasilaky, K., & Zheleva, E. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380255 · Cited by 122
Chen, Z., Xiao, R., Li, C., Ye, G., Sun, H., & Deng, H. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401043 · Cited by 109 · cited 1×
“(3) Regularization has been introduced by some researchers to push the model towards balanced recommendation [32], [33], [34],”
BackgroundNovel Recommendation Based on Personal Popularity Tendency
Oh, J., Park, S., Yu, H., Song, M., & Park, S.-T. (2011) · Proceedings · doi:10.1109/icdm.2011.110 · Cited by 103 · cited 1×
“(3) Regularization has been introduced by some researchers to push the model towards balanced recommendation [32], [33], [34], [35].”
BackgroundProbabilistic Matrix Factorization with Non-random Missing Data
Hernández-Lobato, J. M., Houlsby, N., & Zoubin Ghahramani, A. (2014) · Cited by 101
On a Robust Correlation Coefficient
Abdullah, M. B. (1990) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2349088 · Cited by 98 · cited 5×
“We calculate Kendall Tau Ranking Correlation Coefficient (RCC) [17] between the item lists ranked by them.”
MethodInfluentialAlgorithmic Glass Ceiling in Social Networks
Stoica, A.-A., Riederer, C., & Chaintreau, A. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3186140 · Cited by 95 · cited 1×
“Besides popularity bias, recent works have studied other types of biases in recommendation including: Selection bias, which happens as users are free to choose which items to rate, so that the observed ratings are not a representative sample of all ratings [36], [37], [38], [39]; Exposure bias, which happens in implic…”
BackgroundExploring author gender in book rating and recommendation
Ekstrand, M. D., & Kluver, D. (2021) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-020-09284-2 · Cited by 66
Correcting Popularity Bias by Enhancing Recommendation Neutrality.
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2014) · Cited by 66
Multi-sided Exposure Bias in RecommendationPreprint
Abdollahpouri, H., & Mansoury, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15772 · Cited by 61 · cited 2×
“This phenomenon has been empirically verified by Abdollahpouri [24].”
BackgroundResultSamWalker: Social Recommendation with Informative Sampling Strategy
Chen, J., Wang, C., Zhou, S., Shi, Q., Feng, Y., & Chen, C. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313582 · Cited by 59
Self-citationChen, J., Feng, Y., Ester, M., Zhou, S., Chen, C., & Wang, C. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3271742 · Cited by 38 · cited 1×
“[37], [38], [39]; Exposure bias, which happens in implicit feedback data as users are only exposed to a part of specific items [24], [36], [40], [41]; Position bias, which happens as users tend to interact with items in higher position of the recommendation list [36], [42]; Unfairness [43], [44], which”
BackgroundSelf-citationSocial Recommendation with Missing Not at Random Data
Chen, J., Wang, C., Ester, M., et al. (2018) · Proceedings · doi:10.1109/icdm.2018.00018 · Cited by 36 · cited 1×
“Besides popularity bias, recent works have studied other types of biases in recommendation including: Selection bias, which happens as users are free to choose which items to rate, so that the observed ratings are not a representative sample of all ratings [36], [37], [38], [39]; Exposure bias, which happens in implic…”
BackgroundSelf-citationSerendipitous Personalized Ranking for Top-N Recommendation
Lu, Q., Chen, T., Zhang, W., Yang, D., & Yu, Y. H. (2012) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2012.135 · Cited by 36 · cited 1×
“It would decrease serendipity [25], [26], [27] and fairness [13], [28], [29], [30] of recommendation results, hurting user experience and causing customer churn.”
BackgroundWu, L., He, X., Wang, X., Zhang, K., & Wang, M. (2021) · arXiv (Cornell University) · Cited by 26
Self-citationThe Impact of Popularity Bias on Fairness and Calibration in RecommendationPreprint
Abdollahpouri, H., Mansoury, M., Burke, R., & Mobasher, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05755 · Cited by 18 · cited 1×
“It would decrease serendipity [25], [26], [27] and fairness [13], [28], [29], [30] of recommendation results, hurting user experience and causing customer churn.”
Background
Flagged first, then by citations
1 of the 48 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
“Zheng et al. [217] propose to model personalized conformity effect as users have different sensitivities to the public opinions; while Zhao et al. [215] model time-aware conformity effect by considering item dynamic popularity.”
BackgroundSelf-citationA survey on popularity bias in recommender systems
Klimashevskaia, A., Jannach, D., Elahi, M., & Trattner, C. (2024) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-024-09406-0 · Cited by 223
“[148], for example, found that “the observation data usually exhibits”
BackgroundA survey of sequential recommendation systems: Techniques, evaluation, and future directions
Boka, T. F., Niu, Z., & Neupane, R. B. (2024) · Information Systems · doi:10.1016/j.is.2024.102427 · Cited by 98
Distributionally Robust Graph-based Recommendation System
Wang, B., Chen, J., Li, C., Zhou, S., Shi, Q., Gao, Y., Feng, Y., Chen, C., & Wang, C. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645598 · Cited by 45
Self-citationLalor, J. P., Abbasi, A., Oketch, K., Yang, Y., & Forsgren, N. (2024) · ACM Transactions on Information Systems · doi:10.1145/3641276 · Cited by 40
“For instance, the recommender systems literature has explored the implications of user behavior biases on future model recommendations/performance for many years—namely, the effect of data/sampling biases pertaining to selection, position, exposure, and popularity [34, 64, 75, 108].”
Background
Metadata
- Authors
- Zhao, Z., Chen, J., Zhou, S., He, X., Cao, X., Zhang, F., & Wu, W.
- Affiliations
- University of Science and Technology of China (CN)
- Zhejiang University (CN)
- Meituan (China) (CN)
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2023-10-01
- Volume
- 35
- Issue
- 10
- Pages
- 9920-9931
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W4312551924
- Semantic Scholar
- 3d361ead1582071a5ed26cff391504746a1210a6
- Cited by
- 117
- References
- 48
- Citations by year
- 2026: 21 · 2025: 36 · 2024: 35 · 2023: 21
- FWCI
- 23.75
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Key Research and Development Program of China (2021ZD0111802)
- National Natural Science Foundation of China (62102382, 61972372, U21B2026)
- Meituan Inc.
- CCCD Key Lab of Ministry of Culture and Tourism
- Starry Night Science Fund of Zhejiang University Shanghai Institute for Advanced Study (SN-ZJU-SIAS-001)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- popularity bias, recommender systems, item quality, temporal modeling, disentangled representations, click data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
