The DOI matches its Crossref record; 31 of the 34 references it cites were found in a registry or an index.
Algorithmic Effects on the Diversity of Consumption on Spotify
Open accessAbstract
On many online platforms, users can engage with millions of pieces of content, which they discover either organically or through algorithmically-generated recommendations. While the short-term benefits of recommender systems are well-known, their long-term impacts are less well understood. In this work, we study the user experience on Spotify, a popular music streaming service, through the lens of diversity—the coherence of the set of songs a user listens to. We use a high-fidelity embedding of millions of songs based on listening behavior on Spotify to quantify how musically diverse every user is, and find that high consumption diversity is strongly associated with important long-term user metrics, such as conversion and retention. However, we also find that algorithmically-driven listening through recommendations is associated with reduced consumption diversity. Furthermore, we observe that when users become more diverse in their listening over time, they do so by shifting away from algorithmic consumption and increasing their organic consumption. Finally, we deploy a randomized experiment and show that algorithmic recommendations are more effective for users with lower diversity. Our work illuminates a central tension in online platforms: how do we recommend content that users are likely to enjoy in the short term while simultaneously ensuring they can remain diverse in their consumption in the long term?
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 1×
“To produce such an embedding space, Spotify trains theword2vec algorithm [19] on user-generated playlists.”
Method(2011) · Cited by 4,430
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245 · cited 1×
“Various studies have tested this hypothesis, coming to mixed conclusions about whether, and to what extent, recommender systems display a filter bubble effect [2, 10, 21].”
BackgroundFilter Bubbles, Echo Chambers, and Online News Consumption
Flaxman, S., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw006 · Cited by 1,992 · cited 1×
“Various studies have tested this hypothesis, coming to mixed conclusions about whether, and to what extent, recommender systems display a filter bubble effect [2, 10, 21].”
BackgroundFILTER BUBBLE: Wie wir im Internet entmündigt werden
Pariser, E. (2012) · doi:10.3139/9783446431164 · Cited by 1,849
The Netflix Recommender System: Algorithms, Business Value, and Innovation
Gomez-Uribe, C. A., & Hunt, N. (2016) · ACM Transactions on Management Information Systems · doi:10.1145/2843948 · Cited by 1,418
Solving the apparent diversity-accuracy dilemma of recommender systems
Zhou, T., Kuscsik, Z., Liu, J.-G., Medo, M., Wakeling, J. R., & Zhang, Y.-C. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1000488107 · Cited by 972 · cited 1×
“…filtering framework [1], De Choudhury et al. proposed a clustering technique for recommending social media content that matches a specified level of information diversity [7], and Zhou et al. developed a recommendation algorithm that promotes diversity without sacrificing too much accuracy [32].”
BackgroundNovelty and diversity in information retrieval evaluation
Clarke, C. L. A., Kolla, M., Cormack, G. V., Vechtomova, O., Ashkan, A., Büttcher, S., & MacKinnon, I. (2008) · Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1390334.1390446 · Cited by 962 · cited 1×
“The information retrieval community has been investigating diversity metrics in a number of domains for decades (e.g. see [5, 25]), and more recently, diversity measures have been used to quantify notions of fairness in online platforms [18].”
BackgroundBlockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity
Fleder, D., & Hosanagar, K. (2009) · Management Science · doi:10.1287/mnsc.1080.0974 · Cited by 745 · cited 2×
“There are many definitions of consumption diversity in the literature, the most common of which are functions of the number of times each item is consumed, such as the Gini coefficient [11, 31] and entropy [7, 24].”
BackgroundStructural diversity in social contagion
Ugander, J., Backstrom, L., Marlow, C., & Kleinberg, J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1116502109 · Cited by 695 · cited 1×
“Prior research has shown that diversity is often a desirable property of content consumption, in a variety of contexts [4, 7, 26, 27].”
BackgroundExploring the filter bubble: the effect of using recommender systems on content diversity
Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L., & Konstan, J. A. (2014) · Proceedings of the 23rd international conference on World wide web · doi:10.1145/2566486.2568012 · Cited by 452
The impact of YouTube recommendation system on video views
Zhou, R., Khemmarat, S., & Gao, L. (2010) · Proceedings of the 10th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1879141.1879193 · Cited by 347 · cited 2×
“Fleder and Hosanagar found that in aggregate, diversity decreases as a result of recommender system use in sales [11], whereas Zhou et al. found that the YouTube recommendation system increased the diversity of video views in 2010 [31].”
BackgroundAuralist: introducing serendipity into music recommendation
Zhang, Y. C., Séaghdha, D. Ó., Quercia, D., & Jambor, T. (2012) · Proceedings of the fifth ACM international conference on Web search and data mining · doi:10.1145/2124295.2124300 · Cited by 325
Datta, H., Knox, G., & Bronnenberg, B. J. (2018) · Marketing Science · doi:10.1287/mksc.2017.1051 · Cited by 316 · cited 1×
“conducted a study of music listening behavior across many streaming platforms and found that using streaming services increased the volume and diversity of music that people listened to [6].”
BackgroundMehrotra, R., McInerney, J., Bouchard, H., Lalmas, M., & Diaz, F. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3272027 · Cited by 279 · cited 1×
“see [5, 25]), and more recently, diversity measures have been used to quantify notions of fairness in online platforms [18].”
BackgroundSelf-citationWWW 2014 - Proceedings of the 23rd International Conference on World Wide Web
Nguyen, T. T., Hui, P.-M., Harper, M., Terveen, L., & Konstan, J. A. (2014) · Cited by 146
Stange, K. C. (2009) · The Annals of Family Medicine · doi:10.1370/afm.1003 · Cited by 124 · cited 2×
“This technique stems from a long history of work on generalist and specialist approaches in medicine, the arts, and beyond [3, 26].”
BackgroundChurn prediction in new users of Yahoo! answers
Dror, G., Pelleg, D., Rokhlenko, O., & Szpektor, I. (2012) · International World Wide Web Conference (WWW) · doi:10.1145/2187980.2188207 · Cited by 109
Absence time and user engagement: evaluating ranking functions
Dupret, G., & Lalmas, M. (2013) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2433396.2433418 · Cited by 101
Self-citationReturning is Believing: Optimizing Long-term User Engagement in Recommender Systems
Wu, Q., Wang, H., Hong, L., & Shi, Y. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133025 · Cited by 92 · cited 1×
“Other studies have looked at the relationships between short-term and long-term metrics, e.g. in the context of search [9] and advertising [17], as well as building models that attempt to optimize for both e.g. [16, 29].”
BackgroundOn over-specialization and concentration bias of recommendations
Adamopoulos, P., & Tuzhilin, A. (2014) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2645710.2645752 · Cited by 88
Waller, I., & Anderson, A. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313729 · Cited by 85 · cited 4×
“To quantify musical diversity using our music embedding, we apply the generalist-specialist score, or GS-score [28].”
BackgroundMethodInfluentialSelf-citationPromoting diversity in recommendation by entropy regularizer
Qin, L., & Zhu, X. (2013) · International Joint Conference on Artificial Intelligence · Cited by 74
Identifying relevant social media content: leveraging information diversity and user cognition
Choudhury, M. D., Counts, S., & Czerwinski, M. P. (2011) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/1995966.1995990 · Cited by 70 · cited 3×
“Prior research has shown that diversity is often a desirable property of content consumption, in a variety of contexts [4, 7, 26, 27].”
BackgroundHelping Users Perceive Recommendation Diversity
Hu, R., & Pu, P. (2011) · ACM Conference on Recommender Systems (RecSys) · Cited by 66
Which Diversity Evaluation Measures Are "Good"?
Sakai, T., & Zeng, Z. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331215 · Cited by 43 · cited 1×
“see [5, 25]), and more recently, diversity measures have been used to quantify notions of fairness in online platforms [18].”
BackgroundPromoting Positive Post-Click Experience for In-Stream Yahoo Gemini Users
Lalmas, M., Lehmann, J., Shaked, G., Silvestri, F., & Tolomei, G. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2788581 · Cited by 36 · cited 1×
“in the context of search [9] and advertising [17], as well as buildingmodels that attempt to optimize for both e.”
BackgroundSelf-citationUnderstanding Musical Diversity via Online Social Media
Park, M., Weber, I. G., Naaman, M., & Vieweg, S. E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14620 · Cited by 34 · cited 1×
“Park et al. proposed a definition of musical diversity based on genres and correlated it with demographic and behavioral variables [23].”
BackgroundFocus on the Long-Term: It's better for Users and Business
Hohnhold, H., O’Brien, D., & Tang, D. (2015) · Cited by 10
Encouraging Diversity- and Representation-Awareness in Geographically Centralized Content
Graells-Garrido, E., Lalmas, M., & Baeza-Yates, R. A. (2016) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2856767.2856775 · Cited by 9 · cited 1×
“showing the importance of the interface for users to be aware of diverse content recommended to them [13] and work by Hu and Pu showing that how results are presented influences users’ perceptions of diversity [15].”
BackgroundSelf-citationBeta Survival ModelsPreprint
Hubbard, D., Rostykus, B., Raimond, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03818 · Cited by 2 · cited 1×
“Other studies have looked at the relationships between short-term and long-term metrics, e.g. in the context of search [9] and advertising [17], as well as building models that attempt to optimize for both e.g. [16, 29].”
Background
Research works printed but not linked: 6
- Bublitz Elisabeth, 2014, The skill balancing act: when does broad expertise pay off?Small Business Economics 42, 1
- Flaxman Seth, 2016, S1
- Hohnhold Henning, 2015, Proceedings 21st Conference on Knowledge Discovery and Data Mining
- Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS). Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS).
- Park Minsu, 2015, Ninth International AAAI Conference on Web and Social Media.
- 10.5555/2540128.2540517 · doi:10.5555/2540128.2540517
Cited by
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
“In sum, recommender systems influence cultural evolution by rewiring our social networks and modifying information flows such that they can substantially influence the dynamics of cultural markets 47–49 .”
BackgroundThe Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify
Holtz, D., Carterette, B., Chandar, P., Nazari, Z., Cramer, H. S. M., & Aral, S. (2020) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3391403.3399532 · Cited by 137
Artificial intelligence in the information ecosystem: Affordances for everyday information seeking
Hirvonen, N., Jylhä, V., Lao, Y., & Larsson, S. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24860 · Cited by 107
“…et al., 2020), homog-enize exposure to information (Nechushtai et al., 2023) and taste (see Hesmondhalgh et al., 2023), reduce diversity in content consumption (Anderson et al., 2020), and narrow information landscapes (see Lloyd, 2019) as they prioritize and privilege certain types of information.”
Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation
Gao, C., Huang, K., Chen, J., Zhang, Y., Li, B., Jiang, P., Wang, S., Zhang, Z., & He, X. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591636 · Cited by 120
“A common way of mitigating the Matthew effect in recommendation is to take into account diversity [3, 20, 34, 60].”
BackgroundBello, P., & García, D. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00855-1 · Cited by 60
“According to Spotify, up to one-fifth of their streams can be attributed to algorithmic recommendations (Anderson et al., 2020), which may be enough to sway macrolevel trends in music consumption.”
Background
Metadata
- Authors
- Anderson, A., Maystre, L., Anderson, I., Mehrotra, R., & Lalmas, M.
- Affiliations
- University of Toronto (CA)
- Spotify (Sweden) (SE)
- Venue
- Proceedings of The Web Conference 2020
- Publisher
- ACM
- Published
- 2020-04-20
- Pages
- 2155-2165
- OpenAlex
- W3012950066
- Cited by
- 206
- References
- 34
- Citations by year
- 2026: 22 · 2025: 33 · 2024: 43 · 2023: 42 · 2022: 43 · 2021: 18 · 2020: 5
- FWCI
- 20.48
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- algorithmic recommendation, music streaming, Spotify, recommender systems, user retention, user conversion, randomized experiments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.