Auditing Partisan Audience Bias within Google Search
The DOI matches its Crossref record; 76 of the 77 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
There is a growing consensus that online platforms have a systematic influence on the democratic process. However, research beyond social media is limited. In this paper, we report the results of a mixed-methods algorithm audit of partisan audience bias and personalization within Google Search. Following Donald Trump's inauguration, we recruited 187 participants to complete a survey and install a browser extension that enabled us to collect Search Engine Results Pages (SERPs) from their computers. To quantify partisan audience bias, we developed a domain-level score by leveraging the sharing propensities of registered voters on a large Twitter panel. We found little evidence for the "filter bubble'' hypothesis. Instead, we found that results positioned toward the bottom of Google SERPs were more left-leaning than results positioned toward the top, and that the direction and magnitude of overall lean varied by search query, component type (e.g. "answer boxes"), and other factors. Utilizing rank-weighted metrics that we adapted from prior work, we also found that Google's rankings shifted the average lean of SERPs to the right of their unweighted average.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Research on AI auditing has focused on evaluating whether specific applications meet predefined industry requirements.”
From Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries (2024) · cites this paper as Robertson et al. 2018
In the citing paper
“For instance, researchers have created procedures for auditing AI systems used in recruitment (Kazim et al. 2021), online search (Robertson et al. 2018), and medical diagnostics (Liu et al. 2022).”
From Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries (2024) · cites this paper as Robertson et al. 2018
In the citing paper
“For example, it is well-known that certain users on advertising platforms are more valuable to advertisers than others [48,55,65].”
From Discrimination through Optimization (2019) · cites this paper as [65]
In the citing paper
“Some researchers have leveraged narrow conceptions of auditing to test for bias and discrimination in online ad delivery (Ali et al., 2019;Sweeney, 2013) and autocomplete algorithms (Robertson et al., 2018), for fairness in image classification systems (Morina et al., 2019), for accuracy in news curation systems (Bandy & Diakopoulos, 2019), for completeness in datasets (Coston et al., 2021;Sookhak”
From Auditing of AI: Legal, Ethical and Technical Approaches (2023) · cites this paper as Robertson et al., 2018
Citations
Cites
Birds of a Feather: Homophily in Social Networks
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.415 · Cited by 18,567
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“Spurred by these concerns, research on the spread of fake news on social media has flourished [1, 51, 60, 96].”
BackgroundSocial Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767 · cited 15×
“The resulting bias score is continuous and scales from -1 to 1, where a score of -1 means that a domain i was shared exclusively by Democrats (ri = 0, di > 0), and a score of 1 means that a domain i was shared exclusively by Republicans (ri > 0, di = 0).”
BackgroundMethodResultInfluential(2011) · Cited by 4,430
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
Self-citationProlific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Self-citationExposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 1×
“We utilized two established crowdsourcing services [72, 75] to recruit a demographically diverse sample of 187 participants to take our survey.”
MethodSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
A 61-million-person experiment in social influence and political mobilization
Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012) · Nature · doi:10.1038/nature11421 · Cited by 2,575
Red Media, Blue Media: Evidence of Ideological Selectivity in Media Use
Iyengar, S., & Hahn, K. S. (2009) · Journal of Communication · doi:10.1111/j.1460-2466.2008.01402.x · Cited by 1,940 · cited 1×
“Ancillary evidence for this hypothesis can be found in a number of recent studies, including (1) a 2018 ethnographic study on conservative search engine users which found that they have high levels of trust in Google’s rankings [94], (2) a 2016 study which used poll data from Gallup to show that Republicans trust the…”
ResultPersonal Influence: The Part Played by People in the Flow of Mass Communications
Roshwalb, I., Katz, E., & Lazarsfeld, P. F. (1956) · Journal of Marketing · doi:10.2307/1247718 · Cited by 1,880
FILTER BUBBLE: Wie wir im Internet entmündigt werden
Pariser, E. (2012) · doi:10.3139/9783446431164 · Cited by 1,849
Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips
Sparrow, B., Liu, J., & Wegner, D. M. (2011) · Science · doi:10.1126/science.1207745 · Cited by 1,762 · cited 1×
“When billions of people cognitively depend on an online platform each day [90], every design decision made by the platform carries a broad impact.”
BackgroundThe Fox News Effect: Media Bias and Voting
DellaVigna, S., & Kaplan, E. (2007) · The Quarterly Journal of Economics · doi:10.1162/qjec.122.3.1187 · Cited by 1,602 · cited 3×
“Partisan bias has been shown to influence voting behaviors through newspapers [17, 22, 35], television (e.g., the “Fox News Effect” [20]), social media [10] (see also “digital gerrymandering” [100]), and search engines (e.g., the “Search Engine Manipulation Effect (SEME)” [28, 30]).”
BackgroundWhat Drives Media Slant? Evidence From U.S. Daily Newspapers
(2010) · Econometrica · doi:10.3982/ecta7195 · Cited by 1,483
Carney, D. R., Jost, J. T., Gosling, S. D., & Potter, J. (2008) · Political Psychology · doi:10.1111/j.1467-9221.2008.00668.x · Cited by 1,211
Gillespie, T. (2014) · Media Technologies · doi:10.7551/mitpress/9042.003.0013 · Cited by 1,016
Groseclose, T., & Milyo, J. (2005) · The Quarterly Journal of Economics · doi:10.1162/003355305775097542 · Cited by 991
Messing, S., & Westwood, S. J. (2014) · Communication Research · doi:10.1177/0093650212466406 · Cited by 902
Psychological targeting as an effective approach to digital mass persuasion
Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710966114 · Cited by 892
Shaping the Web: Why the Politics of Search Engines Matters
Introna, L. D. (2000) · The Information Society · doi:10.1080/01972240050133634 · Cited by 783
Shaping the Web: Why the Politics of Search Engines Matters
Introna, L. D., & Nissenbaum, H. (2017) · Computer Ethics · doi:10.4324/9781315259697-19 · Cited by 708
Selective Exposure to Information: A Critical Review
Sears, D. O., & Freedman, J. L. (1967) · Public Opinion Quarterly · doi:10.1086/267513 · Cited by 674
The Structural Virality of Online Diffusion
Goel, S., Anderson, A., Hofman, J., & Watts, D. J. (2016) · Management Science · doi:10.1287/mnsc.2015.2158 · Cited by 647
Media Technologies: Essays on Communication, Materiality, and Society
(2014) · The MIT Press eBooks · doi:10.7551/mitpress/9780262525374.001.0001 · Cited by 580
The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections
Epstein, R., & Robertson, R. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1419828112 · Cited by 577
Self-citationGerber, A. S., Karlan, D., & Bergan, D. (2009) · American Economic Journal: Applied Economics · doi:10.1257/app.1.2.35 · Cited by 577
Himelboim, I., McCreery, S., & Smith, M. (2013) · Journal of Computer-Mediated Communication · doi:10.1111/jcc4.12001 · Cited by 511
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 1×
“• image components that present a collage of image results pulled from Google’s image search feature, some aspects of which have been studied in prior work [21, 55, 91].”
BackgroundLinguistic Models for Analyzing and Detecting Biased Language
Recasens, M. V., Danescu-Niculescu-Mizil, C., & Jurafsky, D. (2013) · Cited by 451
Media Bias and Influence: Evidence from Newspaper Endorsements
Chiang, C.-F., & Knight, B. (2011) · The Review of Economic Studies · doi:10.1093/restud/rdq037 · Cited by 410
MASS MEDIA ATTITUDE CHANGE: Implications of the Elaboration Likelihood Model of Persuasion
(2009) · Media Effects · doi:10.4324/9780203877111-13 · Cited by 409
(2016) · The SAGE Encyclopedia of Corporate Reputation · doi:10.4135/9781483376493.n106 · Cited by 397
Measuring personalization of web search
Hannak, A., Sapiezynski, P., Molavi Kakhki, A., Krishnamurthy, B., Lazer, D., Mislove, A., & Wilson, C. (2013) · Proceedings of the 22nd international conference on World Wide Web · doi:10.1145/2488388.2488435 · Cited by 396
Self-citationThe structure of online diffusion networks
Goel, S., Watts, D. J., & Goldstein, D. G. (2012) · Proceedings of the 13th ACM Conference on Electronic Commerce · doi:10.1145/2229012.2229058 · Cited by 385
Friggeri, A., Adamic, L., Eckles, D., & Cheng, J. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14559 · Cited by 382 · cited 1×
“Today, one-step models have seen a revival of research interest due to the growing capacities of online platforms to target and personalize content [6, 62, 76], and two-step models have been expanded to study digital traces of behavior spreading on social networks [32, 37, 38].”
BackgroundThe Impact of Media Bias: How Editorial Slant Affects Voters
Druckman, J. N., & Parkin, M. (2005) · The Journal of Politics · doi:10.1111/j.1468-2508.2005.00349.x · Cited by 374
Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis
Budak, C., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw007 · Cited by 336 · cited 10×
“…data on the audiences of news outlets with a Facebook presence, were more correlated with the adapted Pew scores (r = 0.97), our measure had stronger correlations with the three other metrics that Ribeiro et al. also used for validation (Facebook [3], AllSides [2], and Budak et al. (2016) [12]).”
BackgroundMethodResultInfluentialThe One-Step Flow of Communication
Bennett, W. L., & Manheim, J. B. (2006) · The ANNALS of the American Academy of Political and Social Science · doi:10.1177/0002716206292266 · Cited by 280
Political Ideology Detection Using Recursive Neural Networks
Iyyer, M., Enns, P., Boyd-Graber, J., & Resnik, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1105 · Cited by 275
Search engine coverage bias: evidence and possible causes
Vaughan, L., & Thelwall, M. (2004) · Information Processing & Management · doi:10.1016/s0306-4573(03)00063-3 · Cited by 259
Personalization of Mass Media and the Growth of Pseudo-Community
BENIGER, J. R. (1987) · Communication Research · doi:10.1177/009365087014003005 · Cited by 255
Models and Insights for Hospital Inpatient Operations: Time-Dependent ED Boarding Time
Shi, P., Chou, M. C., Dai, J. G., Ding, D., & Sim, J. (2016) · Management Science · doi:10.1287/mnsc.2014.2112 · Cited by 175
Personality and attitudes towards current political topics
Riemann, R., Grubich, C., Hempel, S., Mergl, S., & Richter, M. (1993) · Personality and Individual Differences · doi:10.1016/0191-8869(93)90222-o · Cited by 165
Media Bias Monitor: Quantifying Biases of Social Media News Outlets at Large-Scale
Ribeiro, F. N., Henrique, L., Benevenuto, F., Chakraborty, A., Kulshrestha, J., Babaei, M., & Gummadi, K. P. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15025 · Cited by 154
Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages
Robertson, R. E., Lazer, D., & Wilson, C. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186143 · Cited by 139
Self-citationTopical interests and the mitigation of search engine bias
Fortunato, S., Flammini, A., Menczer, F., & Vespignani, A. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0605525103 · Cited by 138
Mass Media Attitude Change: Implications of the Elaboration Likelihood Model of Persuasion
Petty, R. E., Priester, J. R., & Briñol, P. (2002) · Routledge eBooks · doi:10.4324/9781410602428-11 · Cited by 137
Quantifying Political Leaning from Tweets, Retweets, and Retweeters
Wong, F. M. F., Tan, C. W., Sen, S., & Chiang, M. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2553667 · Cited by 136
Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media
Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01347 · Cited by 129
Location, Location, Location: The Impact of Geolocation on Web Search Personalization
Kliman-Silver, C., Hannak, A., Lazer, D., Wilson, C., & Mislove, A. (2015) · Proceedings of the 2015 Internet Measurement Conference · doi:10.1145/2815675.2815714 · Cited by 125
Self-citationJiang, S., & Wilson, C. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274351 · Cited by 125
Self-citationInfluencing Individually: Fusing Personalization and Persuasion
(2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2209310.2209312 · Cited by 119
Dutton, W. H., Reisdorf, B. C., Dubois, E., & Blank, G. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2960697 · Cited by 115
McMahon, C., Johnson, I. L., & Hecht, B. J. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14883 · Cited by 97
Identifying ambiguous queries in web search
Song, R., Luo, Z., Wen, J.-R., Yu, Y. H., & Hon, H.-W. (2007) · International World Wide Web Conference (WWW) · doi:10.1145/1242572.1242749 · Cited by 96
Park, S., Ko, M., Kim, J., Liu, Y., & Song, J. (2011) · Proceedings of the ACM 2011 conference on Computer supported cooperative work · doi:10.1145/1958824.1958842 · Cited by 88
Suppressing the Search Engine Manipulation Effect (SEME)
Epstein, R., Robertson, R. E., Lazer, D., & Wilson, C. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134677 · Cited by 86
Self-citationClassifying the Political Leaning of News Articles and Users from User Votes
Zhou, D. X., Resnick, P., & Mei, Q. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14108 · Cited by 84
Trump vs. Hillary: What Went Viral During the 2016 US Presidential Election
Darwish, K. M., Magdy, W., & Zanouda, T. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67217-5_10 · Cited by 79
Algorithms of Oppression: How Search Engines Reinforce Racism
Hankwitz, M. (2023) · Leonardo · doi:10.1162/leon_r_02331 · Cited by 59
Journalistic Responses to Technological Innovation in Newsrooms
Chadha, K., & Wells, R. (2016) · Digital Journalism · doi:10.1080/21670811.2015.1123100 · Cited by 47
Zittrain, J. L. (2014) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 42
Nyhan, B. J. (2012) · Perspectives on Politics · doi:10.1017/s1537592712001405 · Cited by 39
Political Insights: Exploring partisanship in Web search queries
Borra, E., & Weber, I. G. (2012) · First Monday · doi:10.5210/fm.v17i7.4070 · Cited by 35
Constance: modeling annotation contexts to improve stance classification
Joseph, K., Friedland, L., Hobbs, W., Lazer, D., & Tsur, O. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1116 · Cited by 33 · cited 1×
“Prior work has used this panel matching method to classify political stance and identify individuals whose online discussions accurately characterized election events [47, 52].”
MethodSelf-citationManipulation of Search Engine Results during the 2016 US Congressional Elections
Metaxas, P., & Pruksachatkun, Y. (2017) · Wellesley College Digital RepositoryWellesley (Wellesley College) · Cited by 29
Don’t blame the election on fake news. Blame it on the media.
Watts, D. J., & Rothschild, D. M. (2017) · Columbia journalism review · Cited by 29
Scalable News Slant Measurement Using Twitter
Le, H., Shafiq, Z., & Srinivasan, P. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14957 · Cited by 22
Rosenblum, N. L. (2012) · Perspectives on Politics · doi:10.1017/s1537592712001375 · Cited by 13
Product diffusion through on-demand information-seeking behaviour
Riedl, C., Bjelland, J., Canright, G., Iqbal, A., Engø‐Monsen, K., Qureshi, T., Sundsøy, P., & Lazer, D. M. J. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0751 · Cited by 10
Self-citation"Voters of the Year": 19 Voters Who Were Unintentional Election Poll Sensors on Twitter
Hobbs, W., Friedland, L., Joseph, K., Tsur, O., Wojcik, S., & Lazer, D. M. J. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14920 · Cited by 9
Self-citationTrump vs. Hillary: What went Viral during the 2016 US Presidential ElectionPreprint
Darwish, K. M., Magdy, W., & Zanouda, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03375 · Cited by 8
Research works printed but not linked: 17
- Nicholas J. Belkin. 1980. Anamolous States of Knowledge as a Basis for Information Retrieval. Canadian Journal of Information and Library Sciences 5 (1980), 133--143.
- Nicholas Diakopoulos Daniel Trielli Jennifer Stark and Sean Mussenden. 2018. I Vote For-How Search Informs Our Choice of Candidate. In Digital Dominance: The Power of Google Amazon Facebook and Apple M. Moore and D. Tambini (Eds.). 22.
- William H. Dutton, Bianca Christin Reisdorf, Elizabeth Dubois, and Grant Blank. 2017. Search and Politics: A Cross-National Survey. (2017).
- Google. 2016. Google Trends: Year in Search. Google Trends. (2016). https://trends.google.com/trends/yis/2016/ GLOBAL
- Google. 2017. Google Chrome Privacy Notice. (2017). https://www.google.com/chrome/browser/privacy/ #browser-modes Accessed: 2017-04-01. {42} Jeffrey Gottfried and Elisa Shearer. 2016. News use across social media platforms. Pew Research Center. (2016). http://www.journalism.org/2016/05/26/news-use-across-social-media-platforms-2016/
- Cecilia Kang, Tiffany Hsu, Kevin Roose, Natasha Singer, and Matthew Rosenberg. 2018. Mark Zuckerberg Testimony: Day 2 Brings Tougher Questioning. The New York Times (April 2018).
- Ronald B Mincy. 1993. The Urban Institute audit studies: their research and policy context. Clear and Convincing Evidence: Measurement of Discrimination in America (1993), 165--86.
- Amy Mitchell, Jeffrey Gottfried, Jocelyn Kiley, and Katerina Eva Matsa. 2014. Political Polarization & Media Habits. Pew Research Center's Journalism Project. (Oct. 2014).
- Nic Newman, David A. L. Levy, and Rasmus Kleis Nielsen. 2017. Reuters Institute Digital News Report 2017. SSRN Electronic Journal (2017).
- 10.5555/2029079 · doi:10.5555/2029079
- Media Insight Project. 2014. The Personal News Cycle: How Americans Choose to Get Their News. Media Insight Project. (March 2014). https://www.americanpressinstitute.org/publications/reports/survey-research/ how-americans-get-news/
- Kristen Purcell, Joanna Brenner, and Lee Rainie. 2012. Search engine use 2012. Technical Report. Pew Research Center's Internet and American Life Project.
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. In Proceedings of "Data and Discrimination: Converting Critical Concerns into Productive Inquiry", a Productivereconference at the 64th Annual Meeting of the International Communication Association.
- Jennifer A Stark and Nicholas Diakopoulos. 2017. Using Baselines for Algorithm Audits. In European Conference on Data and Computational Journalism. 3.
- Danny Sullivan. 2016. Google Now Handles at Least 2 Trillion Searches per Year. Search Engine Land. (May 2016). https://searchengineland.com/google-now-handles-2--999-trillion-searches-per-year-250247
- New York Times. 1995. Edward Bernays, 'Father of Public Relations' And Leader in Opinion Making, Dies at 103. (1995).
- Francesca Tripodi. 2018. Searching for Alternative Facts: Analyzing Scriptural Inference in Conservative News Practices. Data & Society. (May 2018).
Cited by
Auditing Large Language Models: A Three-Layered Approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · doi:10.2139/ssrn.4361607 · Cited by 292
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Discrimination through Optimization
Ali, M., Sapiezynski, P., Bogen, M., Korolova, A., Mislove, A., & Rieke, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359301 · Cited by 268
“eotypical gender roles re-enforced by online translation services [12] and image search [43], disparate performance on gender classi˙cation for Black women [13], and political partisanships in search [19, 47, 58]. Although most of the work focused exclusively on the algorithms themselves, recently researchers began to…”
BackgroundMeasuring Misinformation in Video Search Platforms: An Audit Study on YouTube
Hussein, E., Juneja, P., & Mitra, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392854 · Cited by 226
Laine, J., Minkkinen, M., & Mäntymäki, M. (2024) · Information & Management · doi:10.1016/j.im.2024.103969 · Cited by 227
Metadata
- Authors
- Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C.
- Affiliations
- Northeastern University (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2018-11
- Volume
- 2
- Issue
- CSCW
- Pages
- 1-22
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2898883664
- Cited by
- 209
- References
- 76
- Citations by year
- 2026: 18 · 2025: 27 · 2024: 33 · 2023: 31 · 2022: 30 · 2021: 41 · 2020: 13 · 2019: 15 · 2018: 1
- FWCI
- 28.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Funders
- National Science Foundation (IIS-1408345,IIS-1553088)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Communication › Social Media and Politics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- Google search, personalized search, filter bubbles, search result ranking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.