Discrimination in online ad delivery

“, advertisements [29], hotel bookings [17], and image search [13].”

“A related bias is by race, as discovered by technology and government scholar Latanya Sweeney (2013).”

“For example, Sweeney searched more than 2000 racially-associated personal names online and found that ads suggesting arrest were 25% more likely to appear for Black-identifying names (Sweeney 2013).”

“For example, Google’s advertising algorithm exhibits racial bias (Sweeney, 2013).”

“…is inadvertent discrimination: a study (Datta et al., 2015) finds that Google serves fewer ads for high-paying jobs to users profiled as female, and Sweeney (2013) shows that ads for public data about people who are profiled as black are more likely to suggest an arrest record regardless of whether…”

“Similar observations have been made in vision and language models (Zhao et al., 2017), online news (Ross and Carter, 2011), web search (Kay et al., 2015) and advertisements (Sweeney, 2013).”

“Accountability through algorithm can mean reverse-engineering an algorithm to discover how a company used an algorithm to influence the public (Diakopoulos 2013, 2014; Sweeney 2013) or it can mean designing an algorithm that is used to hold decision-makers accountable.”

“…through personalization, young people are encountering a world in which their personal digital data may serve to shape their opportunities and very identities (Friedman & Nissenbaum, 1996; Introna & Nissenbaum, 2000; Pariser, 2012; Sweeney, 2013; Zang, Dummit, Graves, Lisker & Sweeney, 2015).”

“h as gender [130], age [13], or race [11], can endupreceiving verydifferent typesofads.Forexample,searches done with black-sounding names are highly likely to be shown ads suggestive of arrest records [183]. Another study showed that women were shown relatively fewer advertisements for high paying jobs than men with si…”

“[35], or tricks to evade privacy protection mechanisms, like Do-Not-Track signals, are additional tactics that border the tolerance of most users and regulators.”

“applications, such as online reviews (Wallace and Paul, 2016), advertisement (Sweeney, 2013) and web search (Kay et al.”

“Scholarly review points out two major sources of algorithmic problems, which induce algorithm bias—biased training data (Sweeney 2013; O’Neil, 2016) and algorithm design (Obermeyer et al.”

“…men with technical occupations like ‘computer programmer’ and women with domestic occupations like ‘homemaker’), and in online advertising [41] (where Google showed advertisements suggesting that a person had been arrested when that person had a name more often associated with black…”

“In 2013, Latanya Sweeney investigated the placement of online advertisements which implied a given name was associated with a criminal record.[26] She found that the presence of such ads were not the result of particular names being those of criminals, but appeared based on the racial associations of the name, with “b…”

“Separately, Sweeney [63] and Datta et al.”

“One researcher conducted more than 2,000 searches of racially associated personal names across two websites (Sweeney, 2013).”

“Sweeney [76] reported a racial bias being detected in paid results (advertisements) appearing in search results.”

“Emerging research methods for auditing algorithms point in a similar direction (cf. Custers, Calders, Scher-mer, & Zarsky, 2013; Sandvig, Hamilton, Karahalios, & Langbort, 2014; Sweeney, 2013).”

“…studies have shown that Google’s online advertising system displays ads for high-income jobs to men much more often than it does to women [6]; and ads related to arrest records are significantly more likely to show up on searches for distinctively black names or a historically black fraternity [34].”

“Instead, the meaning of the term, and the degree of its realization, are “relational” to the “circumscribed problems” and opportunities of a given community of practice [27, p. 41, 46] such as, a department, or a work group funded through a temporary project.”

“Our work relates to recent efforts to measure various forms of personalization, such as search [21, 47], pricing [31], and ad discrimination [40].”

“1 While there are a few instances of algorithmic audits that have tried to establish whether algorithms are indeed engaged in disparate treatment [25; 83]—and, in fact, find that they are—the vast majority aim to measure what the law calls “disparate”

“Google’s [33] or Facebook’s [2] ad delivery systems generated undesirable discrimination with disparate performance across population groups [9].”

“However, AI systems have been found to perpetuate bias, due to either the unintended consequences of algorithmic design or problematic data (Obermeyer et al. 2019; Richardson et al. 2019; Sweeney 2013).”

“For example, Google search results for stereotypically African American names returned ads containing the word “arrest” significantly more than for white names (Sweeney 2013).”

“We will consider the example of a suspicious advertisement placement based on keyword-matching [129].”

“Many solutions to these problems have been proposed, from technology companies creating jobs centered around ethics to increased auditing of technical systems by researchers to new policy efforts [13, 4, 19, 29, 2].”

“In another, Google searches for names more commonly assigned to Black babies were more likely to display ads for public arrest records than searches for names more commonly assigned to white babies (Sweeney, 2013).”

“If this utility is computed by a machine learning model, an effort must be done to prevent preexisting and technical biases towards a protected group to be embodied in the model (see, e.g. , (Sweeney, 2013)).”

“The unfairness of ML software usually indicates the discriminatory treatment of different groups divided by sensitive features [40], e.g., sex.”

“Documented instances of discriminatory algorithmic decision-making [4, 7, 19, 63] and biased system performance [9, 13, 62, 80] have prompted a growing in-”

“This work shows that although algorithms channel user attention (Hargittai, 2000; Webster, 2014), a lack of algorithmic transparency results in confusion and the potential for discriminatory practices (Introna & Nissenbaum, 2000; Sandvig, Hamilton, Karahalios, & Langbort, 2016; Sweeney, 2013).”

“Moreover, while recent research has insightfully shown how algorithms reinforce racism and mirror racial biases (Noble 2018; Benjamin 2019; Sweeney 2013), this article also shows that the discussion around algorithms and race can go beyond questions of biased algorithms — racism can be found in…”

“…biases such as a tendency to reproduce stereotypic understandings of gender (Crawford & Calo, 2016), interpret Afro-Americans as apes (Hern, 2015), or target certain people with certain recommendations (ads), thereby reproducing and fi xing people in unfavorable social positions (Sweeney, 2013).”

“…such as personalized advertising, recruiting, credit business, or pricing (Dastile et al. 2020; Lambrecht and Tucker 2019; Raghavan et al. 2020; Sweeney 2013), they can gravely impact the further development of peoples’ lives both on the individual and on the societal level, e.g., by…”

“Targeted advertising has come under increasing scrutiny as a threat to privacy and a potential enabler of discrimination based on race, age, sexual orientation, and other sensitive categories [5, 20, 39, 87].”

“In contexts as diverse as hiring [19], lending [4], policing [22], and criminal sentencing [2], mounting empirical evidence suggests these concerns are not merely hypothetical [1, 25].”

“As examples, Sweeney [30] exposed potentially racial bias in the suggestion algo-rithm used in Google AdSense, and Bolukbasi et al. [3] showed the popular word2vec algorithm can be heavily susceptible to gender bias.”

“For instance, an advertising algorithm that targets users based on their web browsing history may inadvertently exclude certain demographic groups that have different online behaviors (Sweeney, 2013).”

“Decisions made in this fashion have long-lasting impact on people’s lives and—absent a careful ethical analysis—may affect certain individuals or social groups negatively [Sweeney, 2013; Angwin et al., 2016; Levin, 2016].”

“Discriminatory harms in algorithmic systems have been examined in technologies as varied as facial recognition (Buolamwini and Gebru, 2018), machine translation (Caliskan et al., 2017), and search engines (Noble, 2018; Sweeney, 2013).”

“…reinforcing negative stereotypes— e.g. , disproportionately suggesting arrest record searches for black-identifying first names in online ad delivery systems [54]—and perpetuating inequities in representation of women and other historically marginalized peoples in different occupational roles [27].”

“Studies showed that online search queries were more likely to return ads from services related to arrest records when names of a particular race were searched than when names of the other race were searched [11, 12].”

“5 In all cases, we focus largely on harms due to over-and under-representation (Kay, Matuszek, and Munson 2015), replicating stereotypes (Sweeney 2013; Caliskan, Bryson, and Narayanan 2017) (particular those Significant limitations..”

“…on the algorithmic infrastructure that sits below this application level (e.g., the algorithms that determine what users see on their Facebook’s news feeds [16, 38], Google’s search algorithms [3], algorithms for predicting likelihood to predict a crime [107], and advertising algorithms [111].”

“Work by Sweeney (2013) identifies representational harms of denigration and stereotyping, where the word “criminal” was more frequently returned in online ads after searches for black-identifying first names.”

“Evolving research highlights the risks of data-driven or algorithmic decision-making [80,81].”

“alized belief about a particular group or class of people [10]. Stereotypes can be positive: Asians are good at math, or negative: African-American names are more associated with criminal backgrounds [35]; Stereotypes are not limited to just racial groups and they can change over time [25]. One of the central purposes…”

“Both first-party and third-party companies tailor a user’s internet experience in part based on these inferences, impacting the search results [47], ads [30, 42], and social feeds [14, 37] a user sees.”

“We take the dataset of female and male (gender), black and white (race) first names used by Sweeney in her Google search bias study (Sweeney, 2013); these names are originally from the studies of Bertrand and Mullainathan (2004), and Fryer and Levitt (Fryer & Levitt, 2004).”

“In addition, there has been an increased reliance of “unintentionally biased” and discriminatory algorithms that support much of the technology that we use today [9, 12].”

“Other well-known examples of digital discrimination are related to race discrimination (Sweeney, 2013; Angwin et al., 2016).”

“…by applications and platforms, and the troubling ends to which this data has sometimes been put, including encouraging addiction (Lewis 2017), enabling discriminatory profiling (O’Neil 2017; Sweeney 2013), mass manipulation of public discourse, and election interference (Rosenberg et al. 2018).”

“The former refers to the circumstances where the algorithm developers, even with good intentions, emphasize certain factors in coding, but end up with unfair results (e.g., online advertisements for arrest records often show up when black names are searched on the web (Sweeney 2013)).”

“Recent reports on data-driven decision making underscore that fairness and equitable treatment of individuals and groups are difficult to achieve [6, 9, 45], and that transparency and accountability of algorithmic processes are indispensable but rarely enacted [12, 17, 57].”

“Decisions made by these algorithms can have a long-lasting impact on people’s lives and may affect certain individuals or social groups negatively [1, 28].”

“For example, in a test for racial bias in Google AdSense, Sweeney (Sweeney 2013) conducted Web searches on names, manipulating them by their racial associations.”

“Sweeney [100], in a test for racial bias in Google AdSense, conducted Web searches on names, manipulating them by their racial associations.”

“Other examples may include stereotypical or prejudicial associations like those highlighted by Sweeney [78] for online ads that associate someone’s name with suggestive ads about arrests.”

“…and discrimination (see Bandy, 2021 for a review), for example, by showing different prices to different people for the same product or service (Hannak et al., 2014), or by personalizing advertisements so that they contain assumptions about criminal records based on a user ’ s name (Sweeney, 2013).”

“In general, any predictive model built on historical data may inadvertently inherit human biases based on gender, ethnicity, race, or religion (Sweeney, 2013; Datta et al., 2015).”

“For example, recently Facebook was questioned regarding bias against women as job ads were targeted toward certain demographic groups and not others (Scheiber, 2018; see also Datta, Tschantz, & Datta, 2015, and Sweeney, 2013, for similar examples).”