SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
The growing volume of digital data stimulates the adoption of recommender systems in different socioeconomic domains, including news industries. While news recommenders help consumers deal with information overload and increase their engagement, their use also raises an increasing number of societal concerns, such as "Matthew effects", "filter bubbles", and the overall lack of transparency. We argue that focusing on transparency for content-providers is an under-explored avenue. As such, we designed a simulation framework called SIREN1 (SImulating Recommender Effects in online News environments), that allows content providers to (i) select and parameterize different recommenders and (ii) analyze and visualize their effects with respect to two diversity metrics. Taking the U.S. news media as a case study, we present an analysis on the recommender effects with respect to long-tail novelty and unexpectedness using SIREN. Our analysis offers a number of interesting findings, such as the similar potential of certain algorithmically simple (item-based k-Nearest Neighbour) and sophisticated strategies (based on Bayesian Personalized Ranking) to increase diversity over time. Overall, we argue that simulating the effects of recommender systems can help content providers to make more informed decisions when choosing algorithmic recommenders, and as such can help mitigate the aforementioned societal concerns.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In each case, attending to end-use provides a richer design space and new opportunities for improvements and safeguards.”
From Toward situated interventions for algorithmic equity: lessons from the field (2020) · cites this paper as [10]
Citations
Cites
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720
(1968) · PubMed · Cited by 4,803
(2011) · Cited by 4,430
Cohen, S. (2011) · Routledge eBooks · doi:10.4324/9780203828250 · Cited by 2,265
FILTER BUBBLE: Wie wir im Internet entmündigt werden
Pariser, E. (2012) · doi:10.3139/9783446431164 · Cited by 1,849
Hindman, M. (2009) · Princeton University Press eBooks · doi:10.1515/9781400837496 · Cited by 1,680
Google news personalization: scalable online collaborative filtering
Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242610 · Cited by 1,511
Content-based book recommending using learning for text categorization
Mooney, R. J., & Roy, L. (2000) · Proceedings of the fifth ACM conference on Digital libraries · doi:10.1145/336597.336662 · Cited by 1,328
Thinking critically about and researching algorithms
Kitchin, R. (2017) · Information, Communication & Society · doi:10.1080/1369118x.2016.1154087 · Cited by 1,186
Personalized news recommendation based on click behavior
Liu, J., Dolan, P., & Pedersen, E. R. (2010) · Proceedings of the 15th international conference on Intelligent user interfaces · doi:10.1145/1719970.1719976 · Cited by 762
Blockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity
Fleder, D., & Hosanagar, K. (2009) · Management Science · doi:10.1287/mnsc.1080.0974 · Cited by 745
Niche news: the politics of news choice
(2011) · Choice Reviews Online · doi:10.5860/choice.49-1894 · Cited by 690
Practical solutions to the problem of diagonal dominance in kernel document clustering
Greene, D., & Cunningham, P. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143892 · Cited by 616
(2022) · Springer eBooks · doi:10.1007/978-1-0716-2197-4 · Cited by 583
Media Technologies: Essays on Communication, Materiality, and Society
(2014) · The MIT Press eBooks · doi:10.7551/mitpress/9780262525374.001.0001 · Cited by 580
Curated Flows: A Framework for Mapping Media Exposure in the Digital Age
Thorson, K., & Wells, C. (2016) · Communication Theory · doi:10.1111/comt.12087 · Cited by 570
Should we worry about filter bubbles?
Zuiderveen Borgesius, F. J., Trilling, D., Möller, J., Bodó, B., de Vreese, C. H., & Helberger, N. (2016) · Internet Policy Review · doi:10.14763/2016.1.401 · Cited by 541
(2017) · Project Muse (Johns Hopkins University) · Cited by 529
Diversity in recommender systems – A survey
Kunaver, M., & Požrl, T. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.02.009 · Cited by 504
Exploring the filter bubble: the effect of using recommender systems on content diversity
Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L., & Konstan, J. A. (2014) · Proceedings of the 23rd international conference on World wide web · doi:10.1145/2566486.2568012 · Cited by 452
Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1444076 · Cited by 387
Breaking the filter bubble: democracy and design
Bozdag, E., & van den Hoven, J. (2015) · Ethics and Information Technology · doi:10.1007/s10676-015-9380-y · Cited by 375
MyMediaLite: a free recommender system library
Gantner, Z., Rendle, S., Freudenthaler, C., & Schmidt-Thieme, L. (2011) · Proceedings of the fifth ACM conference on Recommender systems · doi:10.1145/2043932.2043989 · Cited by 368
Exposure diversity as a design principle for recommender systems
Helberger, N., Karppinen, K., & D’Acunto, L. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2016.1271900 · Cited by 366
Garrett, R. K., Gvirsman, S. D., Johnson, B. K., Tsfati, Y., Neo, R., & Dal, A. (2014) · Human Communication Research · doi:10.1111/hcre.12028 · Cited by 351
Temporal diversity in recommender systems
Lathia, N., Hailes, S., Capra, L., & Amatriain, X. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835486 · Cited by 318
What Do Americans Really Want to Know? Tracking the Behavior of News Readers on the Internet
Tewksbury, D. (2003) · Journal of Communication · doi:10.1093/joc/53.4.694 · Cited by 296
The Platform Press: How Silicon Valley Reengineered Journalism
Bell, E., & Owen, T. (2017) · doi:10.7916/d8r216zz · Cited by 293
Recommendation Networks and the Long Tail of Electronic Commerce1
Oestreicher-Singer, G., & Sundararajan, A. (2012) · MIS Quarterly · doi:10.2307/41410406 · Cited by 278
Modeling and broadening temporal user interest in personalized news recommendation
Li, L., Zheng, L., Yang, F., & Li, T. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.11.020 · Cited by 202
Recommender systems and their impact on sales diversity
Fleder, D. M., & Hosanagar, K. (2007) · Proceedings of the 8th ACM conference on Electronic commerce · doi:10.1145/1250910.1250939 · Cited by 167
WWW 2014 - Proceedings of the 23rd International Conference on World Wide Web
Nguyen, T. T., Hui, P.-M., Harper, M., Terveen, L., & Konstan, J. A. (2014) · Cited by 146
Understanding User Attention and Engagement in Online News Reading
Lagun, D., & Lalmas, M. (2016) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2835776.2835833 · Cited by 122
The Impact of Search and Recommendation Systems on Sales in Electronic Commerce
Hinz, O. F., & Eckert, J. (2010) · Business & Information Systems Engineering · doi:10.1007/s12599-010-0092-x · Cited by 98
‘Machines don’t have instincts’: Articulating the computational in journalism
Bucher, T. (2017) · New Media & Society · doi:10.1177/1461444815624182 · Cited by 96
Investigating serendipity in recommender systems based on real user feedback
Kotkov, D., Konstan, J. A., Zhao, Q., & Veijalainen, J. (2018) · Proceedings of the 33rd Annual ACM Symposium on Applied Computing · doi:10.1145/3167132.3167276 · Cited by 96
Life Cycle Modeling of News Events Using Aging Theory
Chen, C. C., Chen, Y., Sun, Y. S., & Chen, M. C. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39857-8_7 · Cited by 95
Personalized News Recommendation: A Review and an Experimental Investigation
Li, L., Wang, D., Zhu, S., & Li, T. (2011) · Journal of Computer Science and Technology · doi:10.1007/s11390-011-0175-2 · Cited by 91
Kille, B., Hopfgartner, F., Brodt, T., & Heintz, T. (2013) · doi:10.1145/2516641.2516643 · Cited by 91
Tewksbury, D., Hals, M. L., & Bibart, A. (2008) · Journalism & Mass Communication Quarterly · doi:10.1177/107769900808500203 · Cited by 89
Automatic online news topic ranking using media focus and user attention based on aging theory
Wang, C., Zhang, M., Ru, L., & Ma, S. (2008) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1458082.1458219 · Cited by 83
Harambam, J., Helberger, N., & van Hoboken, J. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0088 · Cited by 79
Self-citationLeckner, S. (2012) · Visual Communication · doi:10.1177/1470357211434029 · Cited by 65
Personalized Ranking for non-uniformly sampled items
Gantner, Z., Drumond, L., Freudenthaler, C., & Schmidt-Thieme, L. (2011) · Cited by 62
Nadolski, R. J., van den Berg, B., Berlanga, A. J., Drachsler, H., Hummel, H. G. K., Koper, R., & Sloep, P. (2008) · DSpace (Open University in the Netherlands) · Cited by 57
An Unexpectedness-Augmented Utility Model for Making Serendipitous Recommendation
Zheng, Q., Chan, C.-K., & Ip, H. H. S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20910-4_16 · Cited by 55
THE FOURTH ESTATE IDEAL IN JOURNALISM HISTORY
(2009) · The Routledge Companion to News and Journalism · doi:10.4324/9780203869468-7 · Cited by 54
The choice of digital newspapers: influence of reader goals and user experience
Flavián, C., & Gurrea, R. (2006) · Internet Research · doi:10.1108/10662240610673673 · Cited by 49
Novelty and diversity enhancement and evaluation in Recommender Systems
Vargas, S. (2015) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 48
Deep Neural Networks for News Recommendations
Park, K., Lee, J., & Choi, J. (2017) · doi:10.1145/3132847.3133154 · Cited by 47
How Americans encounter, recall and act upon digital news
Mitchell, A., Gottfried, J. A., Shearer, E., & Lu, K. (2017) · Cited by 42
Factor models for tag recommendation in bibsonomy
Rendle, S., & Schmidt-Thieme, L. (2009) · Cited by 35
Karlsson, M. (2016) · Observatorio (OBS*) · doi:10.15847/obsobs1042016940 · Cited by 30
A Serendipity Model for News Recommendation
Jenders, M., Lindhauer, T., Kasneci, G., Krestel, R., & Naumann, F. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24489-1_9 · Cited by 27
(2011) · Choice Reviews Online · doi:10.5860/choice.48-6565 · Cited by 23
Construction of an Ideal Map (IdMap) based on the ideal profiles obtained directly from consumers
Worch, T., Lê, S., Punter, P. H., & Pagès, J. (2012) · Food Quality and Preference · doi:10.1016/j.foodqual.2012.04.003 · Cited by 23
When Deliberation Divides: Processes Underlying Mobilization to Collective Action
Wojcieszak, M. (2011) · Communication Monographs · doi:10.1080/03637751.2011.589459 · Cited by 19
Designing a Web-based Testing Tool for Multi-Criteria Recommender Systems
Manouselis, N., & Costopoulou, C. I. (2006) · Cited by 15
Sie, R. L. L., Bitter-Rijpkema, M., & Sloep, P. (2010) · Procedia Computer Science · doi:10.1016/j.procs.2010.08.015 · Cited by 14
Evaluation of Collaborative Filtering by Agent-Based Simulation Considering Market Environment
Umeda, T., Ichikawa, M., Koyama, Y., & Deguchi, H. (2009) · Developments in Business Simulation and Experiential Learning: Proceedings of the Annual ABSEL conference · Cited by 8
A method for evaluating discoverability and navigability of recommendation algorithms
Lamprecht, D., Strohmaier, M., & Helić, D. (2017) · Computational Social Networks · doi:10.1186/s40649-017-0045-3 · Cited by 5
Research works printed but not linked: 7
- Gantner Zeno, 2012, Proceedings of KDD Cup
- Manouselis Nikos, 2007, Proceedings of the Workshop on Social Information Retrieval for Technology Enhanced Learning. 27--35
- Mitchell Amy, 2016, Long-form reading shows signs of life in our mobile news world
- Newman Nic, 2017, David AL Levy, and Rasmus Kleis Nielsen
- Rendle Steffen, 2009, ECML/PKDD 2008 Discovery Challenge Workshop, part of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. 235--243
- 10.1111/j.1460-2466.2003.tb02918.x · doi:10.1111/j.1460-2466.2003.tb02918.x · Paper page
- Zuckerman Ethan, Digital cosmopolitans: Why we think the Internet connects us, why it doesn't, and how to rewire it
Cited by
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153
“Other scholars have pursued empirical case studies [19, 50, 52], field evaluations [10, 65], context-aware uses of datasets [46, 63], and questions of institutional access to real-world data [67].”
BackgroundEthical and regulatory challenges of Generative AI in education: a systematic review
García-López, I. M., & Trujillo-Liñán, L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1565938 · Cited by 144
Recommender systems effect on the evolution of users’ choices distribution
Hazrati, N., & Ricci⋆, F. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102766 · Cited by 56
Fair ranking: a critical review, challenges, and future directions
Patro, G. K., Porcaro, L., Mitchell, L., Zhang, Q., Zehlike, M., & Garg, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533238 · Cited by 110
News recommender systems: a programmatic research review
Mitova, E., Blassnig, S., Strikovic, E., Urman, A., Hannák, A., de Vreese, C. H., & Esser, F. (2022) · Annals of the International Communication Association · doi:10.1080/23808985.2022.2142149 · Cited by 82
“Scholars have, however, highlighted the importance of comparing different platforms and filtering mechanisms (e.g. Bountouridis et al., 2019; Haim et al., 2018; Jürgens & Stark, 2022; Möller et al., 2018).”
BackgroundInfluential
Metadata
- Authors
- Bountouridis, D., Harambam, J., Makhortykh, M., Marrero, M., Tintarev, N., & Hauff, C.
- Affiliations
- Delft University of Technology (NL)
- University of Amsterdam (NL)
- ORCID
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 150-159
- OpenAlex
- W2909551589
- Cited by
- 92
- References
- 61
- Citations by year
- 2026: 11 · 2025: 30 · 2024: 19 · 2023: 7 · 2022: 8 · 2021: 10 · 2020: 6 · 2019: 1
- FWCI
- 4.94
- Type
- conference paper
- Language
- en
- Open access
- green
- resolver.tudelft.nl
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- Research Repository (Delft University of Technology) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- recommender systems, online news, simulation framework, diversity indices, unexpectedness, content providers, transparency, filter bubbles, Matthew effect, Bayesian Personalized Ranking, k-nearest neighbors, information overload
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.