Building Human Values into Recommender Systems: An Interdisciplinary Synthesis
Open accessThe DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
Abstract
Recommender systems are the algorithms which select, filter, and personalize content across many of the world's largest platforms and apps. As such, their positive and negative effects on individuals and on societies have been extensively theorized and studied. Our overarching question is how to ensure that recommender systems enact the values of the individuals and societies that they serve. Addressing this question in a principled fashion requires technical knowledge of recommender design and operation, and also critically depends on insights from diverse fields including social science, ethics, economics, psychology, policy, and law. This article is a multidisciplinary effort to synthesize theory and practice from different perspectives, with the goal of providing a shared language, articulating current design approaches, and identifying open problems. We collect a set of values that seem most relevant to recommender systems operating across different domains, and then examine them from the perspectives of current industry practice, measurement, product design, and policy approaches. Important open problems include multi-stakeholder processes for defining values and resolving trade-offs, better values-driven measurements, recommender controls that people use, non-behavioral algorithmic feedback, optimization for long-term outcomes, causal inference of recommender effects, academic-industry research collaborations, and interdisciplinary policy-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Multidimensional Scale of Perceived Social Support
Zimet, G. D., Dahlem, N. W., Zimet, S. G., & Farley, G. K. (1988) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5201_2 · Cited by 13,339
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 2×
“This explainable-by-design approach avoids the challenges of interpreting learned models (Rudin, 2019), but generating understandable explanations from deep learning models is an active area of research that may yet prove fruitful (Kim et al., 2018; Kokhlikyan et al., 2020; Sundararajan et al.,…”
BackgroundMethodThe structure of psychological well-being revisited.
Ryff, C. D., & Keyes, C. L. M. (1995) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.69.4.719 · Cited by 10,285
Ellison, N. B., Steinfield, C., & Lampe, C. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00367.x · Cited by 9,939 · cited 1×
“Social media use has been associated with increased social connection and social capital in online and offline social networks, with particular benefit for users experiencing low self-esteem and low life-satisfaction (Ellison et al., 2007; Steinfield et al., 2008; Subrahmanyam et al., 2008).”
BackgroundA meta-analytic test of intergroup contact theory.
Pettigrew, T. F., & Tropp, L. R. (2006) · Journal of personality and social psychology · doi:10.1037/0022-3514.90.5.751 · Cited by 9,708 · cited 1×
“In the case of diversity, designers may research the history of attempts to create inter-group tolerance through diversity of exposure, including the ways in which this has failed (Paolini et al., 2010; Pettigrew and Tropp, 2006).”
BackgroundThe spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“For example, some evidence suggests fabricated news articles (as identified by third-party fact checkers) spread significantly faster through Twitter than genuine news articles, especially articles about politics (Vosoughi et al., 2018).”
BackgroundSubjective well-being. The science of happiness and a proposal for a national index
Diener, E. (2000) · American Psychologist · doi:10.1037/0003-066x.55.1.34 · Cited by 7,121 · cited 1×
“More recently there has been increasing focus on a more holistic understanding of well-being based on both subjective and objective measures (Diener, 2000; Hicks et al., 2013; Krueger and Stone, 2014).”
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
The WHO-5 well-being index: A systematic review of the literature
Topp, C. W., Østergaard, S. D., Søndergaard, S., & Bech, P. (2015) · Psychotherapy and Psychosomatics · doi:10.1159/000376585 · Cited by 4,708
Collective action and the evolution of social norms.
Ostrom, E. (2000) · Journal of Economic Perspectives · doi:10.1257/jep.14.3.137 · Cited by 4,195
An overview of the Schwartz theory of basic values
Schwartz, S. H. (2012) · Online Readings in Psychology and Culture · doi:10.9707/2307-0919.1116 · Cited by 4,147 · cited 2×
“Entertainment can also contribute to well-being, especially as a short-term emotional experience, and hedonism is often considered a basic drive (Schwartz, 2012).”
BackgroundBaron-Cohen, S., & Wheelwright, S. (2004) · Journal of Autism and Developmental Disorders · doi:10.1023/b:jadd.0000022607.19833.00 · Cited by 4,093
Validation of a New General Self-Efficacy Scale
Chen, G., Gully, S. M., & Eden, D. (2001) · Organizational Research Methods · doi:10.1177/109442810141004 · Cited by 3,542 · cited 2×
“General Self-Efficacy Scale (GSE) Ex: “It is easy for me to stick to my aims and accomplish my goals” (Chen et al., 2001) Autonomy Scale (AS): Measures components of autonomy including family loyalty autonomy, value autonomy, emotional autonomy, and behavioral autonomy (Anderson et al.,…”
BackgroundA 61-million-person experiment in social influence and political mobilization
Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012) · Nature · doi:10.1038/nature11421 · Cited by 2,575
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 2×
“While companies can and do perform internal algorithm audits of various types (Chowdhury, 2021; Raji et al., 2020) regulation could require external audits of recommender systems to mitigate these concerns (Singh and Doty, 2021a).”
BackgroundValenzuela, S., Park, N., & Kee, K. F. (2009) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2009.01474.x · Cited by 2,229 · cited 1×
“In one study of over 2,000 college students, social media use was associated with improvement in various facets of psychological well-being such as overall life satisfaction, civic engagement, and social trust (Valenzuela et al., 2009).”
BackgroundThe challenge of defining wellbeing
Dodge, R., Daly, A., Huyton, J., & Sanders, L. (2012) · International Journal of Wellbeing · doi:10.5502/ijw.v2i3.4 · Cited by 2,166
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
(2020) · doi:10.1145/3313831 · Cited by 1,951
Social capital, self-esteem, and use of online social network sites: A longitudinal analysis.
Steinfield, C., Ellison, N. B., & Lampe, C. (2008) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2008.07.002 · Cited by 1,919 · cited 1×
“Social media use has been associated with increased social connection and social capital in online and offline social networks, with particular benefit for users experiencing low self-esteem and low life-satisfaction (Ellison et al., 2007; Steinfield et al., 2008; Subrahmanyam et al., 2008).”
BackgroundExposure to opposing views on social media can increase political polarization
Bail, C. A., Argyle, L. P., Brown, T. W., Bumpus, J. P., Chen, H., Hunzaker, M. B. F., Lee, J., Mann, M., Merhout, F., & Volfovsky, A. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1804840115 · Cited by 1,863 · cited 1×
“In particular, it will be important to test for backfire effects where people reject diverse information (Bail et al., 2018; Dunn and Singh, 2014) or other unintended consequences.”
BackgroundCertifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01056-1 · Cited by 1,800 · cited 2×
“…COVID-19 misinfo has been common on social media globally, especially early in the pandemic (Al-Zaman, 2021; Brennen et al., 2020) and there is evidence that suggests misinformation has contributed to the spread of the virus via reduced vaccination rates (Loomba et al., 2021; Pinna et al., 2021).”
BackgroundRoutledge International Handbook of Participatory Design
(2012) · doi:10.4324/9780203108543 · Cited by 1,606
Online and offline social networks: Use of social networking sites by emerging adults
Subrahmanyam, K., Reich, S. M., Waechter, N., & Espinoza, G. (2008) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2008.07.003 · Cited by 1,284 · cited 1×
“Social media use has been associated with increased social connection and social capital in online and offline social networks, with particular benefit for users experiencing low self-esteem and low life-satisfaction (Ellison et al., 2007; Steinfield et al., 2008; Subrahmanyam et al., 2008).”
BackgroundKim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
Do social network sites enhance or undermine subjective well-being? A critical review
Verduyn, P., Ybarra, O., Résibois, M., Jonides, J., & Kross, E. (2017) · Social Issues and Policy Review · doi:10.1111/sipr.12033 · Cited by 1,209
Response bias, social desirability and dissimulation
Furnham, A. (1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90014-0 · Cited by 1,033
Fjeld, J., Achten, N., Hilligoss, H., Nagy, A., & Srikumar, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3518482 · Cited by 998 · cited 6×
“…and family life” (Council of Europe, 2020) Users should be allowed to determine if and how their personal data is collected, processed and disseminated (Fjeld et al., 2020, p. 21; IEEE, 2021, p. 71) GDPR Data Protection Impact Assessment (European Commission, 2016) Controls to limit data use.”
BackgroundMethodInfluentialFighting misinformation on social media using crowdsourced judgments of news source quality
Pennycook, G., & Rand, D. G. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806781116 · Cited by 947 · cited 2×
“, 2019) plus (ii) a wide variety of scoring signals that range from source credibility (Pennycook and Rand, 2019) to playlist diversity (Mehrotra et al.”
BackgroundTechnology and the Virtues: A Philosophical Guide to a Future Worth Wanting
Vallor, S. (2016) · doi:10.1093/acprof:oso/9780190498511.001.0001 · Cited by 910
Przybylski, A. K., & Weinstein, N. (2017) · Psychological Science · doi:10.1177/0956797616678438 · Cited by 890
Kashdan, T. B., Rose, P., & Fincham, F. D. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8203_05 · Cited by 888
The welfare effects of social media
Allcott, H., Braghieri, L., Eichmeyer, S., & Gentzkow, M. (2020) · American Economic Review · doi:10.1257/aer.20190658 · Cited by 857 · cited 4×
“In another study, deactivating social media accounts for four weeks resulted in increased time in offline interactions and improved subjective well-being (Allcott et al., 2020).”
BackgroundInfluentialLambrecht, A., & Tucker, C. (2019) · Management Science · doi:10.1287/mnsc.2018.3093 · Cited by 854 · cited 1×
“Recommender systems used to promote job openings may be discriminatory if they do not consider the balance of distribution across legally protected user attributes (Lambrecht and Tucker, 2019).”
BackgroundExplainable Recommendation: A Survey and New Perspectives
Zhang, Y., & Chen, X. (2020) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000066 · Cited by 755
Bias and Debias in Recommender System: A Survey and Future Directions
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., & He, X. (2023) · ACM Transactions on Information Systems · doi:10.1145/3564284 · Cited by 741 · cited 2×
“” “Fairness” or “bias” is primarily concerned with the distribution of benefits and harms between people or groups (Chen et al., 2020; M. D. Ekstrand et al., 2021).”
BackgroundAre people incidentally exposed to news on social media? A comparative analysis
Fletcher, R., & Nielsen, R. K. (2018) · New Media & Society · doi:10.1177/1461444817724170 · Cited by 738
Inspiration as a psychological construct.
Thrash, T. M., & Elliot, A. J. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.4.871 · Cited by 705
How to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671
Linton, M.-J., Dieppe, P., & Medina-Lara, A. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010641 · Cited by 638
McCoy, J., & Somer, M. (2019) · The ANNALS of the American Academy of Political and Social Science · doi:10.1177/0002716218818782 · Cited by 595
Auditing radicalization pathways on YouTube
Ribeiro, M. H., Ottoni, R., West, R., Almeida, V. A. F., & Meira, W. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372879 · Cited by 580
Recommender systems and their ethical challenges
Milano, S., Taddeo, M., & Floridi, L. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00950-y · Cited by 556 · cited 3×
“Previous work has explored the legal and human rights potentially impacted by recommender systems (Milano et al., 2020) and by AI in general (Donahoe and Metzger, 2019)”
BackgroundShould we worry about filter bubbles?
Zuiderveen Borgesius, F. J., Trilling, D., Möller, J., Bodó, B., de Vreese, C. H., & Helberger, N. (2016) · Internet Policy Review · doi:10.14763/2016.1.401 · Cited by 541 · cited 1×
“This sort of optimization-driven shift has been frequently suggested as a mechanism driving filter bubble, echo chamber, or polarization effects, though the empirical evidence is mixed (Bruns, 2019; Guess et al., 2018; Stray, 2021a; Zuiderveen Borgesius et al., 2016).”
BackgroundSelf-citationA Survey on Conversational Recommender Systems
Jannach, D., Manzoor, A., Cai, W., & Chen, L. (2022) · ACM Computing Surveys · doi:10.1145/3453154 · Cited by 521
The General Belongingness Scale (GBS): Assessing achieved belongingness
Malone, G. P., Pillow, D. R., & Osman, A. (2012) · Personality and Individual Differences · doi:10.1016/j.paid.2011.10.027 · Cited by 517
Nabi, R. L., Prestin, A., & So, J. (2013) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2012.0521 · Cited by 510 · cited 1×
“Larger social networks may be associated with greater perceived social support, reduced stress, and improved well-being (Nabi et al., 2013).”
BackgroundMeasuring Social Norms and Preferences Using Experimental Games: A Guide for Social Scientists
Camerer, C. F., & Fehr, E. (2004) · Foundations of Human Sociality · doi:10.1093/0199262055.003.0003 · Cited by 465
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448 · cited 1×
“Transparency can be important for building users’ trust (Cramer et al., 2008) and is often discussed as a policy tool to promote understanding and accountability.”
BackgroundHow digital media drive affective polarization through partisan sorting
Törnberg, P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2207159119 · Cited by 438
Yarchi, M., Baden, C., & Kligler-Vilenchik, N. (2021) · Political Communication · doi:10.1080/10584609.2020.1785067 · Cited by 437
Underspecification Presents Challenges for Credibility in Modern Machine Learning
D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.03395 · Cited by 398
A Survey of Value Sensitive Design Methods
Friedman, B., Hendry, D. G., & Borning, A. (2017) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000015 · Cited by 389 · cited 1×
“Therefore, building human values into recommender systems requires a mix of conceptual, empirical, and technical work (Friedman et al., 2017).”
BackgroundMöller, J., Trilling, D., Helberger, N., & van Es, B. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1444076 · Cited by 387
Self-citationOn the Democratic Role of News Recommenders
Helberger, N. (2019) · Digital Journalism · doi:10.1080/21670811.2019.1623700 · Cited by 384 · cited 6×
“Diversity has been most specifically studied in the context of news recommendations, where it might serve a variety of democratic goals including consumer choice, civic participation, pluralist tolerance, or challenging the status quo (Bernstein et al., 2020; Helberger, 2019; Vrijenhoek et al., 2020).”
BackgroundInfluentialSelf-citationAssociation of Facebook Use With Compromised Well-Being: A Longitudinal Study
Shakya, H. B., & Christakis, N. A. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kww189 · Cited by 378 · cited 1×
“One longitudinal study compared social media use with mental health, physical health, and found a decrease of 5%-8% of a standard deviation in self-reported mental health (Shakya and Christakis, 2017).”
BackgroundNext steps for value sensitive design
Borning, A., & Muller, M. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208560 · Cited by 365
Jacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362 · cited 1×
“Jacobs and Wallach (2021) propose several other types of validity in the context of measurements within computational systems, including face validity (is the metric plausible?)”
BackgroundCross-Country Trends in Affective Polarization
Boxell, L., Gentzkow, M., & Shapiro, J. (2020) · doi:10.3386/w26669 · Cited by 359
A systematic review of worldwide causal and correlational evidence on digital media and democracy
Lorenz-Spreen, P., Oswald, L., Lewandowsky, S., & Hertwig, R. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01460-1 · Cited by 348
Equity of Attention: Amortizing Individual Fairness in Rankings
Biega, A. J., Gummadi, K. P., & Weikum, G. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210063 · Cited by 337
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336
Like-minded sources on Facebook are prevalent but not polarizing
Nyhan, B., Settle, J., Thorson, E., Wojcieszak, M., Barberá, P., Chen, A. Y., Allcott, H., Brown, T., Crespo-Tenorio, A., Dimmery, D., et al. (2023) · Nature · doi:10.1038/s41586-023-06297-w · Cited by 333
Bekalu, M. A., McCloud, R. F., & Viswanath, K. (2019) · Health Education & Behavior · doi:10.1177/1090198119863768 · Cited by 303 · cited 2×
“Some investigators have suggested that the amount of time spent online is less important than the quality of that time; active use may promote well-being, whereas passive use and emotional connection to use may have a negative impact (Bekalu et al., 2019; Frison and Eggermont, 2020; Verduyn et al., 2017).”
BackgroundDeep Reinforcement Learning in Large Discrete Action Spaces
Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., & Coppin, B. (2015) · arXiv · doi:10.48550/arxiv.1512.07679 · Cited by 290
Pennycook, G., & Rand, D. G. (2022) · Nature Communications · doi:10.1038/s41467-022-30073-5 · Cited by 270
Right-Wing YouTube: A Supply and Demand Perspective
Munger, K., & Phillips, J. (2022) · The International Journal of Press/Politics · doi:10.1177/1940161220964767 · Cited by 265
Cultural Heritage and Human Rights
Silverman, H., & Ruggles, D. F. (2007) · Cultural Heritage and Human Rights · doi:10.1007/978-0-387-71313-7_1 · Cited by 256 · cited 2×
“” (Silverman and Ruggles, 2007) Percentage of locally produced content.”
Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
Hase, P., & Bansal, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.491 · Cited by 254
The challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253 · cited 1×
“From a technical perspective, explanations can be tricky to generate because many recommender techniques (including deep learning) do not lend themselves to easy explanations (Weld and Bansal, 2019).”
BackgroundFrison, E., & Eggermont, S. (2020) · Communication Research · doi:10.1177/0093650215617506 · Cited by 239
Nechushtai, E., & Lewis, S. C. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.043 · Cited by 237
Degenerate Feedback Loops in Recommender Systems
Jiang, R., Chiappa, S., Lattimore, T., György, A., & Kohli, P. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314288 · Cited by 232 · cited 3×
“A polarizing preference shift effect has been demonstrated in multiple simulations with different specifications (Carroll et al., 2021; Evans and Kasirzadeh, 2021; Jiang et al., 2019; Kalimeris et al., 2021; Krueger et al., 2020; Rychwalska and Roszczyńska-Kurasińska, 2018) which suggests that it could be a robust eff…”
BackgroundIs the Internet Causing Political Polarization? Evidence from Demographics
Boxell, L., Gentzkow, M., & Shapiro, J. (2017) · doi:10.3386/w23258 · Cited by 226 · cited 4×
“, polarization began increasing decades before social media (Boxell et al., 2017) while several other developed countries have similar internet usage but do not show increasing polarization (Boxell et al.”
BackgroundInfluentialValue-Sensitive Algorithm Design: Method, Case Study, and Lessons
Zhu, H., Yu, B., Halfaker, A., & Terveen, L. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274463 · Cited by 220
White, J. B., Langer, E. J., Yariv, L., & Welch, J. C. (2006) · Journal of Adult Development · doi:10.1007/s10804-006-9005-0 · Cited by 207
Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
Bandy, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449148 · Cited by 193
The Online Social Support Scale: Measure development and validation.
Nick, E. A., Cole, D. A., Cho, S.-J., Smith, D. K., Carter, T. G., & Zelkowitz, R. L. (2018) · Psychological Assessment · doi:10.1037/pas0000558 · Cited by 191
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Interactive collaborative filtering
Zhao, X., Zhang, W., & Wang, J. (2013) · Proceedings of the 22nd ACM international conference on Information & Knowledge Management · doi:10.1145/2505515.2505690 · Cited by 165 · cited 4×
“The tools of interactive optimization and utility elicitation (Boutilier et al., 2003; Zhao et al., 2013) could play an important role here, though these approaches have not yet found widespread use in practical recommender design.”
BackgroundInfluentialWhen every day is a high school reunion: Social media comparisons and self-esteem.
Midgley, C., Thai, S., Lockwood, P., Kovacheff, C., & Page-Gould, E. (2021) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000336 · Cited by 164
Social choice ethics in artificial intelligence
Baum, S. D. (2020) · AI & SOCIETY · doi:10.1007/s00146-017-0760-1 · Cited by 158 · cited 1×
“Hence, social choice approaches cannot excuse system designers from making consequential normative choices (Baum, 2020).”
BackgroundArtificial Intelligence and Human Rights
Donahoe, E., & Metzger, M. M. (2019) · Journal of Democracy · doi:10.1353/jod.2019.0029 · Cited by 158
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153
Recommender systems and the amplification of extremist content
Whittaker, J., Looney, S., Reed, A., & Votta, F. (2021) · Internet Policy Review · doi:10.14763/2021.2.1565 · Cited by 135 · cited 3×
“This has generally shown that engaging with some type of content increases its frequency (Wall Street Journal, 2021; Whittaker et al., 2021) but this design models users as unchanging, so it does not provide evidence on persuasive effects.”
BackgroundInfluentialAndrus, M., Spitzer, E., Brown, J., & Xiang, A. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445888 · Cited by 111
Self-citationPutting Fairness Principles into Practice: Challenges, Metrics, and Improvements
Beutel, A., Chen, J., Doshi, T., Qian, H., Woodruff, A., Luu, C., Kreitmann, P., Bischof, J., & Chi, E. H. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314234 · Cited by 111
The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality
Gordon, M. L., Zhou, K., Patel, K., Hashimoto, T., & Bernstein, M. S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445423 · Cited by 101
Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists
Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., & Wilson, C. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317595 · Cited by 92 · cited 1×
“A wide variety of fairness metrics concerning the exposure of items, groups of items, or producers have been proposed, though many of these are closely related (Kuhlman et al., 2021; Raj and Ekstrand, 2022; Sapiezynski et al., 2019; Zehlike et al., 2022).”
BackgroundCategorizing Variants of Goodhart's Law
Manheim, D., & Garrabrant, S. (2018) · arXiv · doi:10.48550/arxiv.1803.04585 · Cited by 73
The Illusion of Control: Placebo Effects of Control Settings
Vaccaro, K., Huang, D., Eslami, M., Sandvig, C., Hamilton, K., & Karahalios, K. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173590 · Cited by 65
Algorithmic Amplification of Politics on Twitter
Huszár, F., Ktena, S. I., O’Brien, C., Belli, L., Schlaikjer, A., & Hardt, M. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2025334119 · Cited by 65
IEEE 7010: A New Standard for Assessing the Well-being Implications of Artificial Intelligence
Schiff, D., Ayesh, A., Musikanski, L., & Havens, J. C. (2020) · 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc42975.2020.9283454 · Cited by 62 · cited 3×
“Collections of AI-relevant metrics such as the IEEE 7010 standard (Schiff et al., 2020) provide useful compendiums, but measures developed in social science and policy may not apply directly to particular recommender contexts such as news recommendation or targeted advertising, and will need to be refined.”
BackgroundInfluentialMore Accounts, Fewer Links: How Algorithmic Curation Impacts Media Exposure in Twitter Timelines
Bandy, J., & Diakopoulos, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449152 · Cited by 54
The Challenge of Value Alignment: From Fairer Algorithms to AI Safety
Gabriel, I., & Ghazavi, V. (2022) · Oxford Handbook of Digital Ethics · doi:10.1093/oxfordhb/9780198857815.013.18 · Cited by 52
Choice Set Misspecification in Reward Inference
Freedman, R., Shah, R., & Dragan, A. (2021) · arXiv · doi:10.48550/arxiv.2101.07691 · Cited by 6 · cited 1×
“Further, someone might be uninformed, coerced, addicted, have lowered expectations (Anderson, 2001) or want something that isn’t available (Freedman et al., 2020; Robertson and Salehi, 2020).”
Background
Cited by
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
“Selection occurs when certain cultural traits are more likely to be adopted by individuals due to factors such as their usefulness,”
BackgroundSelf-citationACROCPoLis: A Descriptive Framework for Making Sense of Fairness
Aler Tubella, A., Coelho Mollo, D., Dahlgren Lindström, A., Devinney, H., Dignum, V., Ericson, P., Jonsson, A., Kampik, T., Lenaerts, T., Mendez, J. A., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594059 · Cited by 8
“This opacity has triggered studies into the features of the algorithms as well as the consequences of their operation within the platform, based on publicly accessible facts [3, 52].”
Background(2024) · doi:10.1145/3630106.3659004 · Cited by 3
Metadata
- Authors
- Stray, J., Halevy, A., Assar, P., Hadfield-Menell, D., Boutilier, C., Ashar, A., Bakalar, C., Beattie, L., Ekstrand, M., Leibowicz, C., Moon Sehat, C., Johansen, S., Kerlin, L., Vickrey, D., Singh, S., Vrijenhoek, S., Zhang, A., Andrus, M., Helberger, N., Proutskova, P., Mitra, T., & Vasan, N.
- Venue
- ACM Transactions on Recommender Systems
- Published
- 2024-06-05
- Volume
- 2
- Issue
- 3
- Pages
- 1-57
- ISSN
- 2770-6699 (ISSN-L)
- OpenAlex
- W4388631551
- Cited by
- 122
- References
- 96
- FWCI
- 28.67
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.