Discrimination in online ad delivery

“This is shown, for example, by profiling algorithms that inadvertently discriminate against marginalised populations (Barocas and Selbst, 2015; Birrer, 2005), as seen in delivery of online advertisements according to perceived ethnicity (Sweeney, 2013).”

“Documented cases of automated prejudice range from online advertising (Sweeney, 2013) to criminal sentencing (Angwin et al., 2016).”

“A number of online systems have been shown to exhibit various biases, such as racial discrimination and gender bias in the ads presented to users [36, 6].”

“Second, in a very different domain, researchers have begun to analyze the ways in which different genders and racial groups experience advertising and commercial content on the Internet differently [9, 26].”

“Discrimination is a key concern when it comes to biased AI systems, as they can perpetuate and even amplify existing inequalities [24].”

“Work by Sweeney (2013) identifies representational harms of denigration and stereotyping , where the word “criminal” was more frequently returned in online ads after searches for black-identifying first names.”

“Such issues have been demonstrated in various learning systems, including online advertisement systems (Sweeney, 2013), word embedding models (Bolukbasi et al.”

“The researchers should deliberate over not only the benefits of research, but also over the possible types of harms (Barocas et al., 2017), the affected groups, and how to test for adverse impact (Sweeney, 2013).”

“I analyzed the cases presented above, together with the interviews of key informants and a review of algorithm reverse engineering in the literature (Baker and Potts 2013; Guha, Cheng, and Franci 2010; Lazer et al. 2014; Mikians et al. 2012; Mukherjee et al. 2013; Hannak et al. 2013; Sweeney 2013).”

“At the same time, scholars and activists have pointed to a range of social, ethical and legal issues associated with algorithmic decision-making, including bias and discrimination [4,63], and lack of transparency and accountability [15,47,70,69].”

“Source code for AdFisher, raw data, and additional details about our experiments can be found at http://www.cs.cmu.edu/ ∼ mtschant/ife/”

“Similar observations have been made in vision and language models (Zhao et al., 2017), online news (Ross and Carter, 2011), web search (Kay et al., 2015) and advertisements (Sweeney, 2013).”

“ng systems in a wide variety of applications, ranging from content recommendations to pretrial risk assessment, has raised concerns about their potential unfairness towards people with certain traits [8, 22, 24, 27]. Anti-discrimination laws in various countries prohibit unfair treatment of individuals based on specic…”

“The key benefits of algorithmic transparency and interpretability include: user awareness [9]; bias and discrimination detection [45, 196]; interpretable behavior of intelligent systems [124]; and accountability for users [46].”

“In the United States, the Obama White House ‒ presaged by key early scholarship (e.g., Citron, 2007; Sweeney, 2013) and developments overseas (e.g., European Commission, 2012) ‒ confronted the challenges of so-called ‘ big data ’ through the lens of discrimination and civil rights (Podesta,…”

“Both algorithms and interfaces, however, can be biased in how they represent the world [9,34].”

“Among the distinct anxieties caused by AI, bias behavior anxiety concerns AI ’ s ability to accurately recognize differences between individuals [31,58].”

“Personalised platforms build detailed profiles of their user’s attributes and behaviour, which determine the content they view, the products they see and search results they receive (Tufekci, 2014; Sweeney, 2013).”

“, advertisements [75], hotel bookings [53], and image search [45].”

“Modern online ads are individually targeted, but often based on race-based correlations; corporations often excuse such biases on the basis that the bias was in the data [118].”

“In general, any predictive model built on historical data may inadvertently inherit human biases based on gender, ethnicity, race, or religion (Sweeney, 2013; Datta et al., 2015).”

“Building on important germinal scholarship [10]-[12], these reports framed concerns over discrimination”

“Stereotype Analysis Implicit stereotypes have been observed in applications such as online advertising systems (Sweeney, 2013), web search (Kay et al.”

“And this concern has not been without reason: a steady stream of empirical findings has shown that data-driven methods can unintentionally both encode existing human biases and introduce new ones.(7,9,11,60) At the same time, the last two years have seen an unprecedented explosion in interest from the academic communi…”

“Previous work has investigated how many algorithms rely on data that is imbued with societal biases and explored how to address this issue [3, 12, 50, 56].”

“It turns out that the answer is positive: for instance, recent studies in the context of online advertising show that ads for high-income jobs are presented to men much more often than to women [5]; and ads for arrest records are significantly more likely to show up on searches for distinctively black names [16].”

“Gender bias has been shown to affect several realworld applications relying on automatic language analysis, including online news (Ross and Carter, 2011), advertisements (Sweeney, 2013), abusive language detection (Park et al.”

“…borrowers based on the borrower’s social network connections’’ (Sullivan 2015) and online advertisers can use racial stereotypes to create targeted ads (Sweeney 2013)—but to criticize or resist such predictions means understanding how algorithms create and associate categories like ‘‘friends…”

“This is often a point of introduction for problems related to bugs and biases, and is well described in the literature (e.g., Sweeney 2013; Datta et al. 2015; O’Neil 2016; Boscoe 2019).”

“For example, Latanya Sweeney found that Google served ads that disparaged African Americans [58], while Datta et al.”

“Separately, Sweeney [63] and Datta et al.”

“Existing instances of research using these methods have found important discriminatory features in algorithmically mediated platforms, including racial discrimination in online advertising delivery (Sweeney 2013) and price discrimination on e-commerce websites (Hannak et al. 2014; Diakopoulos 2013).”

“…the case for automated text coding, where systematically flawed human judgment can deform the performance of learning algorithms, and, a fortiori , especially if algorithms incorporate, however inadvertently, judgments based on irrational prejudice (Barocas and Selbst, forthcoming; Sweeney, 2013).”

“…and statistics have been increasingly applied throughout a wide range of real-world decision-making scenarios, including in healthcare, law enforcement, education, and finance (Mahoney and Mohen 2007; Brennan, Dieterich, and Ehret 2009; Khandani, Kim, and Lo 2010; Sweeney 2013; Angwin et al. 2016).”

“study of online ads showed that women were considerably less likely to receive ads for high-income jobs (Datta, Tschantz, & Datta, 2015), while another study showed that searching for names associated with African Americans generated a higher proportion of adds that suggested arrest (Sweeney, 2013).”

“Therefore, since ML is a new field, many norms and regulations may not yet have been explored or defined [60, 86].”

“Decisions made by these algorithms can have a long-lasting impact on people’s lives and may affect certain individuals or social groups negatively [1, 28].”

“HCI, CSCW, and other computing communities examine, design, and deploy varied technologies based on machine learning, which perpetuates discrimination, forms of racial profling, and exploitation [1, 16, 38, 44, 82, 93] and thus must consider inclusive methods that might eliminate harmful practices.”

“This may intensify existing representational and allocative harms from search systems [37, 45].”

“than those predominantly given to white babies [186], and how facial recognition systems tend to misclassify darker-skinned women more than other groups [19].”

“And even ad recommenders can exhibit racial discrimination [26].”

“67 Sweeney 2013.”

“mputing devices to swi›ly replicate decision-making can lead to magnifying erroneous behavior [9]. Barocas and Selbst [6], Citron and Pasquale [12], Edelman and Luca [24], Kay et al. [42] and Sweeney [65] show that algorithms can make discriminatory decisions even if the computing process is well-intentioned. „is is b…”

“Many now-classic bodies of work study discrimination and harms caused by machine learning systems on historically disadvantaged groups in settings ranging from ad delivery [48] to facial analysis [8] to search engine bias [40] and provision of public services [13].”

“Some researchers have leveraged narrow conceptions of auditing to test for bias and discrimination in online ad delivery (Ali et al., 2019; Sweeney, 2013) and autocomplete algorithms (Robertson et al., 2018), for fairness in image classification systems (Morina et al., 2019), for accuracy in news…”

“, age correlates with savings, race correlates with name, and, in the United States, race correlates with zip code, models trained without access to a set of attributes can still effectively act unfairly with respect to those attributes [113, 114].”

“, 2016] and other work has focused on algorithmic decision-making, including the auto-complete function of search engines [Baker and Potts, 2013], advertisements based on search terms [Sweeney, 2013], and image search results [Kay et al.”

“Want to upload a video to YouTube and have closed captions generated automatically? If your video features a man’s voice, the closed captions are likely to be more accurate than one with a woman’s voice [65].”

“mender system may unfavorably recommend fewer job opportunities to individuals of a certain gender [16] or individuals in an underrepresented ethnic group [17].”

“Theoretically, algorithms have great potential to reform existing systems to become both more ecient and equitable, but as exposed by various highprole investigations [2, 11, 23, 27], prediction-based models that make or assist with consequential decisions are, in practice, highly prone to reproducing past and current…”

“creating discriminatory ads based on gender [7] or race [38], showing different prices for the same products/services to different users [15] and mistakenly labeling a black man as an ape by an image tagging algorithm [16] – are some such examples.”

“This includes examining broad classes of systems like search [14, 14, 25, 28, 32, 35, 36, 41, 53–55], e-commerce [10, 26], news recommendation [4, 31], online advertising [58, 60], maps [57], ridesharing [9, 30], online reviews and ratings [17, 18], natural language processing [7], and recommendation [29].”

“Historically, machine learning models have uninten - tionally perpetuated societal stereotypes and discriminated against specific groups based on gender, race, age, or other demographic factors [ 112, 113].”

“Employment Display job openings based on user profiles Job opportunities not offered based on traditional biases (Sweeney 2013; Savage 2016)”

“Source code for AdFisher, raw data, and additional details about our experiments can be found at http://www.cs.cmu.edu/ ∼ mtschant/ife/”

“Growing evidence suggests (Barocas and Selbst 2016; Kay et al. 2015; Edelman and Luca 2014; Citron and Pasquale 2014; Sweeney 2013) that decision making by inappropriately trained algorithms can discriminate people.”

“…implicitly associated with sex offenders (Ananny, 2011), African-Americans who are suggested as having an arrest record (Barocas and Selbst, 2016; Sweeney, 2013), and women whose pregnancies are predicted based on their purchase histories (Baumer, 2015; Duhigg, 2012)—would not be included among…”

“A situation in which the failure to remove bias in an application leads to undesirable outcomes comes from Sweeney [21], who analyzed ads provided by Google, and came to the conclusion that for certain searches " a black-identifying name was 25% more likely to get an ad suggestive of an arrest…”

“Together, these watchdogs continue to participate in consequential studies to analyze the biased outcomes of models used in domains as varied as content curation [80], online advertising [40], tenant screening [39, 70], child welfare [12], criminal justice [5, 10, 20, 26, 78], hiring [63], immigration [24], education…”

“Sweeney (2013) asks, 5 ‘Why is this discrimination occurring?”

“For identity-coded names, we use the first and last names that are distinctive for each intersectional group according to the US Census, and avoid names with notable figures 71–73 .”

“The foundational work of Sweeney [66] and Noble [54], among others, has documented how modern search engines absorb and amplify biases and then reflect them back to users, showing a world where, for example, Blackness is associated with criminality and”

“Discriminating ads have been delivered based on racialized search results [51] and gendered user profiles [15].”

“For example, [27] has demonstrated racial bias in Google’s AdSense system that were only made visible by looking at behavior across multiple users and sessions.”

“Under these constructs, any individual or institution can be prejudiced, biased or bigoted, but only members of the dominant social groups or institutions can be ‘racist’ given their capability to exert power over others (Sweeney, 2013).”

“Google ads have been found to be racist, raising the question of how algorithms work, and whether (and how) their training material should be reconsidered (Sweeney, 2013).”

“Unfortunately, evidence has been found suggesting that negative discrimination (such as racism) might be performed in online ad delivery [58].”

“Sweeney [76] show through ad data that advertisements related to arrest records were more likely to be displayed when searching for “Black-sounding” names, which may affect a candidate’s employment capability, for example.”

“Recently, the rise of algorithmic platforms’ influence on users’ online experience has motivated many studies (Datta et al. 2015; Sweeney 2013; Hannak et al. 2014; Sandvig et al. 2014; Executive Office of the President 2016) to audit these platforms and understand their biases.”

“on based on information available on user proles [55], social feedback by the community as a manifestation of oine discrimination in online context and algorithms reinforcing existing gender biases [53, 45, 34]. Scholars are also drawing attention to the legal aspects of discrimination and labor market protections in…”

“As another example, search queries including stereotypically African-American names ( e.g. , Latanya or Latisha) vs. stereotypically white names ( e.g. , Kristen or Jill) result in a higher chance of seeing advertisements that suggest the person has an arrest record [82].”

“For example, Sweeney discovered systemic bias in Google AdSense, an algorithmic advertising platform [21].”