The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
How algorithmic confounding in recommendation systems increases homogeneity and decreases utility
Open accessAbstract
Recommendation systems are ubiquitous and impact many domains; they have the potential to influence product consumption, individuals' perceptions of the world, and life-altering decisions. These systems are often evaluated or trained with data from users already exposed to algorithmic recommendations; this creates a pernicious feedback loop. Using simulations, we demonstrate how using data confounded in this way homogenizes user behavior without increasing utility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Jolliffe, I. (2005) · Encyclopedia of Statistics in Behavioral Science · doi:10.1002/0470013192.bsa501 · Cited by 14,310
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,673
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Lee, D. D., & Seung, H. S. (2000) · neural information processing systems · Cited by 9,877
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced Textbooks in Control and Signal Processing · doi:10.1007/978-1-4471-0347-9_4 · Cited by 6,594
Evaluating collaborative filtering recommender systems
Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Riedl, J. T. (2004) · ACM Transactions on Information Systems · doi:10.1145/963770.963772 · Cited by 5,798 · cited 1×
“Accuracy metrics are popular because they are straightforward to understand and easy to implement, but still they do not necessarily capture the usefulness of recommendations; there are a variety of system characteristics that are thought to be important, along with methods for evaluating them [24].”
BackgroundBlei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Probabilistic Matrix Factorization
Mnih, A., & Salakhutdinov, R. (2007) · UCL Discovery (University College London) · Cited by 4,517
The MovieLens Datasets: History and Context
Harper, F. M., & Konstan, J. A. (2016) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2827872 · Cited by 3,907
Factorization meets the neighborhood: a multifaceted collaborative filtering model
Koren, Y. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401944 · Cited by 3,852
(2011) · Springer eBooks · doi:10.1007/978-0-387-85820-3 · Cited by 3,761
Collaborative Filtering for Implicit Feedback Datasets
Hu, Y., Koren, Y., & Volinsky, C. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.22 · Cited by 3,170
The long tail: why the future of business is selling less of more
(2007) · Choice Reviews Online · doi:10.5860/choice.44-2783 · Cited by 2,869
A contextual-bandit approach to personalized news article recommendation
Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772758 · Cited by 2,562
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Due to these and other concerns, regulations are emerging to restrict automated individual decision-making, such as recommendation systems [21]; this work will aid in making such eorts eective.”
BackgroundRecommender systems with social regularization
Ma, H., Zhou, D., Liu, C., Lyu, M. R., & King, I. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935877 · Cited by 1,626
Collaborative topic modeling for recommending scientific articles
Wang, C., & Blei, D. M. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020480 · Cited by 1,527
Performance of recommender algorithms on top-n recommendation tasks
Cremonesi, P., Koren, Y., & Turrin, R. (2010) · Proceedings of the fourth ACM conference on Recommender systems · doi:10.1145/1864708.1864721 · Cited by 1,410
Bennett, J. R., & Lanning, S. (2007) · Cited by 1,216
Collaborative filtering with temporal dynamics
Koren, Y. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557072 · Cited by 1,203
Trust-aware recommender systems
Massa, P., & Avesani, P. (2007) · Proceedings of the 2007 ACM conference on Recommender systems · doi:10.1145/1297231.1297235 · Cited by 1,190
Solving the apparent diversity-accuracy dilemma of recommender systems
Zhou, T., Kuscsik, Z., Liu, J.-G., Medo, M., Wakeling, J. R., & Zhang, Y.-C. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1000488107 · Cited by 972
Counterfactual reasoning and learning systems: the example of computational advertising
Bottou, L., Peters, J., Quiñonero-Candela, J., Charles, D. X., Chickering, D. M., Portugaly, E., Ray, D., Simard, P., & Snelson, E. (2013) · Journal of Machine Learning Research · Cited by 863
Discrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“Previous work has investigated how many algorithms rely on data that is imbued with societal biases and explored how to address this issue [3, 12, 50, 56].”
BackgroundCollaborative filtering with temporal dynamics
Koren, Y. (2010) · Communications of the ACM · doi:10.1145/1721654.1721677 · Cited by 776
Blockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity
Fleder, D., & Hosanagar, K. (2009) · Management Science · doi:10.1287/mnsc.1080.0974 · Cited by 745
Lessons from the Netflix prize challenge
Bell, R. M., & Koren, Y. (2007) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1345448.1345465 · Cited by 727
A new similarity measure for collaborative filtering to alleviate the new user cold-starting problem
Ahn, H. J. (2008) · Information Sciences · doi:10.1016/j.ins.2007.07.024 · Cited by 712
Controlled experiments on the web: survey and practical guide
Kohavi, R., Longbotham, R., Sommerfield, D., & Henne, R. M. (2009) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-008-0114-1 · Cited by 690
Guo, G., Zhang, J., & Yorke-Smith, N. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9153 · Cited by 573
MovieLens unplugged: experiences with an occasionally connected recommender system
Miller, B. N., Albert, I., Lam, S. K., Konstan, J. A., & Riedl, J. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604094 · Cited by 508
Sweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Kaminskas, M., & Bridge, D. (2017) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2926720 · Cited by 465
Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms
Li, L., Chu, W., Langford, J., & Wang, X. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935878 · Cited by 443
The long tail of recommender systems and how to leverage it
Park, Y.-J., & Tuzhilin, A. (2008) · Proceedings of the 2008 ACM conference on Recommender systems · doi:10.1145/1454008.1454012 · Cited by 436
Biased assimilation, homophily, and the dynamics of polarization
Dandekar, P., Goel, A., & Lee, D. T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1217220110 · Cited by 435
Blei, D. (2011) · Proceedings of the 17th ACM SIGKDD International Conference Tutorials · doi:10.1145/2107736.2107741 · Cited by 424
Modeling User Exposure in Recommendation
Liang, D., Charlin, L., McInerney, J., & Blei, D. M. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883090 · Cited by 362
Collaborative prediction and ranking with non-random missing data
Marlin, B. M., & Zemel, R. S. (2009) · Proceedings of the third ACM conference on Recommender systems · doi:10.1145/1639714.1639717 · Cited by 315
Evaluation of recommendations: rating-prediction and ranking
Steck, H. (2013) · Proceedings of the 7th ACM conference on Recommender systems · doi:10.1145/2507157.2507160 · Cited by 256
Biased Assimilation, Homophily, and the Dynamics of Polarization
Dandekar, P., Goel, A., & Lee, D. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35311-6_47 · Cited by 201
GaP: a factor model for discrete data
Canny, J. (2004) · Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1008992.1009016 · Cited by 178
From hits to niches?: or how popular artists can bias music recommendation and discovery
Celma, Ò., & Cano, P. (2008) · Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition · doi:10.1145/1722149.1722154 · Cited by 170
A Probabilistic Model for Using Social Networks in Personalized Item Recommendation
Chaney, A. J. B., Blei, D. M., & Eliassi-Rad, T. (2015) · Proceedings of the 9th ACM Conference on Recommender Systems · doi:10.1145/2792838.2800193 · Cited by 164
Self-citationLeveraging Missing Ratings to Improve Online Recommendation Systems
Ying, Y., Feinberg, F., & Wedel, M. (2006) · Journal of Marketing Research · doi:10.1509/jmkr.43.3.355 · Cited by 162
Learning to rank for recommender systems
Karatzoglou, A., Baltrunas, L., & Shi, Y. (2013) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2507157.2508063 · Cited by 143
Diversity in Big Data: A Review
Drosou, M., Jagadish, H. V., Pitoura, E., & Stoyanovich, J. (2017) · Big Data · doi:10.1089/big.2016.0054 · Cited by 140 · cited 1×
“less useful than a collection of diverse items [19].”
BackgroundEstimating the Causal Impact of Recommendation Systems from Observational Data
Sharma, A. K., Hofman, J. M., & Watts, D. J. (2015) · ACM Conference on Economics and Computation (EC) · doi:10.1145/2764468.2764488 · Cited by 130
A probabilistic model to resolve diversity–accuracy challenge of recommendation systems
Javari, A., & Jalili, M. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-014-0779-2 · Cited by 103
Data and algorithmic bias in the web
Baeza-Yates, R. A. (2016) · ACM Web Science Conference (WebSci) · doi:10.1145/2908131.2908135 · Cited by 94
Exploring automatic music annotation with "acoustically-objective" tags
Tingle, D., Kim, Y. E., & Turnbull, D. (2010) · Multimedia Information Retrieval · doi:10.1145/1743384.1743400 · Cited by 89
The Effect of Recommendations on Network Structure
Su, J., Sharma, A., & Goel, S. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883040 · Cited by 78 · cited 1×
“Some work has also been done to understand the causal impact of these systems on behavior by nding natural experiments in observational data [53, 55] (approximating expensive controlled experiments [33]), but it is unclear howwell these results generalize.”
BackgroundExplore-exploit in top-N recommender systems via Gaussian processes
Vanchinathan, H. P., Nikolic, I., Bona, F. D., & Krause, A. (2014) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2645710.2645733 · Cited by 76
Solving the accuracy-diversity dilemma via directed random walks
Liu, J.-G., Shi, K., & Guo, Q. (2012) · Physical Review E · doi:10.1103/physreve.85.016118 · Cited by 60
The Long Tail in Recommender Systems
Celma, Ò. (2010) · Springer eBooks · doi:10.1007/978-3-642-13287-2_4 · Cited by 22
Reducing Offline Evaluation Bias in Recommendation SystemsPreprint
Myttenaere, A. D., Grand, B. L., Golden, B., & Rossi, F. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.0822 · Cited by 21
Modeling mutual feedback between users and recommender systems
Zeng, A., Yeung, C. H., Medo, M., & Zhang, Y. (2015) · reroDoc Digital Library · Cited by 13
Cited by
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741
“Many researchers also study the impact of feedback loop on the popularity bias [57], [58], [59].”
BackgroundCausal Intervention for Leveraging Popularity Bias in Recommendation
Zhang, Y., Feng, F., He, X., Wei, T., Song, C., Ling, G., & Zhang, Y. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462875 · Cited by 443
Disentangling User Interest and Conformity for Recommendation with Causal Embedding
Zheng, Y., Gao, C., Li, X., He, X., Li, Y., & Jin, D. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449788 · Cited by 356
“Real-world recommender systems are often trained and updated continuously using real-time user interactions, which forms a feedback loop as illustrated in Figure 1, with training data and test data NOT independent and identically distributed (IID) [13].”
MethodEvolution and impact of bias in human and machine learning algorithm interaction
Sun, W., Nasraoui, O., & Shafto, P. (2020) · PLOS ONE · doi:10.1371/journal.pone.0235502 · Cited by 116
Metadata
- Authors
- Chaney, A. J. B., Stewart, B. M., & Engelhardt, B. E.
- Venue
- Proceedings of the 12th ACM Conference on Recommender Systems
- Publisher
- ACM
- Published
- 2018-09-27
- Pages
- 224-232
- OpenAlex
- W2765564115
- Cited by
- 331
- References
- 58
- FWCI
- 18.14
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.