Discrimination through Optimization

In the paper

“Furthermore, recent work has demonstrated that fairness is not necessarily composable, i.e., for several notions of fairness such as individual fairness [24], a collection of classifiers that are fair in isolation do not necessarily result in a fair outcome when they are used as part of a larger system [25].”

What the sources say

• Supported

  The Abstract (p. 1) says that classifiers fair in isolation do not necessarily compose into fair systems. The Conclusions (p. 65) explicitly apply this result to individual fairness and a large class of group fairness definitions. (LLM, conf=0.95)

  cites [25] 25. Cynthia Dwork and Christina Ilvento. 2018. Fairness Under Composition. In 10th Innovations in Theoretical Computer Science Conference (ITCS 2019) (Leibniz International Proceedings in Informatics (LIPIcs)), Vol. 124. 33:1-33:20. https: //doi.org/10.4230/LIPIcs.ITCS.2019.33 source

In the paper

“Most platforms have a review process to prevent abuse or violations of their platforms' advertising policies[8,77].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [8] 8. About Twitter Ads approval [n.d.]. About Twitter Ads Approval. https://business.twitter.com/en/help/ads-policies/ introduction-to-twitter-ads/about-twitter-ads-approval.html.
• ?Could not verify

  The available Abstract discusses auditing data brokers through Facebook advertising, but does not address ad-review processes. Searches for review processes, ad approval, and advertising-policy violations found no passage supporting the claim that most platforms have such a process; the accessible copy appears to contain mainly the abstract, references, and page metadata. (LLM, conf=0.72)

  cites [77] 77. Giridhari Venkatadri, Piotr Sapiezyński, Elissa M. Redmiles, Alan Mislove, Oana Goga, Michelle Mazurek, and Krishna P. Gummadi. 2019. Auditing Offline Data Brokers Via Facebook's Advertising Platform. In International World Wide Web Conference. San Francisco, California, USA. sourceExperimental — accuracy not yet measured

In the paper

“Google, Twitter, and Pinterest all provide similar features; these are called Customer Match[7], Tailored Audiences, and Customer Lists[61], respectively.”

What the sources say

• ?Could not verify

  The Hiring Websites section (page not provided) discusses resume-search services, not advertising audience features. Searches of our full-text copy for Customer Lists, Tailored Audiences, Customer Match, and Pinterest found no passage addressing the attributed feature names. (LLM, conf=0.76)

  cites [7] 7. About Customer Match [n.d.]. About Customer Match. https://support.google.com/adwords/answer/6379332?hl=en. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [61] 61. New Marketing API Requirements for all Advertising Campaigns [n.d.]. New Marketing Api Re- quirements For All Advertising Campaigns. https://developers.facebook.com/blog/post/2019/08/15/ new-marketing-api-requirements-for-all-advertising-campaigns/.

In the paper

“Google and Pinterest offer similar features: on Google it is called Similar Audiences[40], and on Pinterest it is called Actalike Audiences[63].”

What the sources say

• ?Not checked — no full text of the sourcecites [40] 40. Muhammad Ali, et al. source
• ?Could not verify

  The citation to this paper is attached to the Google portion of the sentence, not the Pinterest portion. The Results (p. 19) discuss Google's ad-platform tracking, and User-Targeting Objectives (p. 4) discusses interest-based targeting, but neither identifies a Google feature called “Similar Audiences.” Searches for that name and for Google lookalike-audience targeting found no supporting passage in our copy. (LLM, conf=0.70)

  cites [63] 63. Javier Parra-Arnau, Jagdish Prasad Achara, and Claude Castelluccia. 2017. Myadchoices: Bringing Transparency And Control To Online Advertising. ACM Transactions on the Web 11 (2017). sourceExperimental — accuracy not yet measured

In the paper

“Similarly, ads referring to cultural content stereotypically of most interest to Black users (e.g., hip-hop) can deliver to over 85% Black users, and those referring to content stereotypically of interest to white users (e.g., country music) can deliver to over 80% white users, even when targeted identically by the advertiser.”

In the paper

“In the most extreme cases, our ads for jobs in the lumber industry reach an audience that is 72% white and 90% male, our ads for cashier positions in supermarkets reach an 85% female audience, and our ads for positions in taxi companies reach a 75% Black audience, even though the targeted audience specified by us as an advertiser is identical for all three.”

In the paper

“This fraction remains fairly consistent across our experiments regardless of what we advertise, thus introducing the same amount of noise across our measurements.”

In the paper

“However, as we show with ads targeting all U.S. users, internal competition does not appear to influence the results.”

In the paper

“ads targeting the same audience but that include a creative that would stereotypically be of the most interest to men (e.g., bodybuilding) can deliver to over 80% men, and those that include a creative that would stereotypically be of the most interest to women (e.g., cosmetics) can deliver to over 90% women.”

In the paper

“We find that both the advertiser's budget and the content of the ad each significantly contribute to the skew of Facebook's ad delivery.”

In the paper

“The data we obtained has 7,560,885 individuals, with 93% reporting their race as either Black or White.”

In the paper

“Interestingly, we do not observe a significant change in users' behavior to explain this swap: the bottom graph plots the click through rates (CTRs) for both ads by men and women over time.”

In the paper

“Notably, we observe that the ad with an image of bodybuilding but cosmetics text delivers closest to 50:50 across genders, but the ad with the image of cosmetics but bodybuilding text does not.”

In the paper

“In both cases, we observe that changes in ad delivery due to differences in budget are indeed happening: the higher the daily budget, the smaller the fraction of men in the audience, with the Pearson's correlation of ρ = -0.88,p val < 10 -5 for all U.S. users and ρ = -0.73,p val < 10 -3 for the custom audiences.”

• Also stated in: § 5 CONCLUDING DISCUSSION · boosted

  “• Replicated and affirmed prior research suggesting that market and pricing dynamics can create conditions that lead to differential outcomes, by showing that the lower the daily budget for an ad, the fewer women it is delivered to.”

In the paper

“We observe significant ad delivery skew along racial lines in the delivery of our ads, with certain ads delivering to an audience of over 72% Black users (comparable to the baseline results) while others delivering to an audience of as little as 51% Black users.”

In the paper

“In particular, the bodybuilding ad ended up being delivered to over 75% men on average, while the cosmetics ad ended up being delivered to over 90% women on average.”

In the paper

“In Figure 6B we observe that the ad delivery differences are, indeed, even greater: the male visible and invisible images deliver to approximately 60% men, while the female visible and invisible images deliver to approximately 45% men.”

In the paper

“Neutral ads are seen by a relatively balanced, 45% white audience, while the audiences receiving the country and hip-hop ads are 80% and 13% white, respectively.”

In the paper

“We can immediately observe drastic differences in ad delivery across our ads along both racial and gender lines: our five ads for positions in the lumber industry deliver to over 90% men and to over 70% white users in aggregate, while our five ads for janitors deliver to over 65% women and over 75% black users in aggregate.”

• Also stated in: § 5 CONCLUDING DISCUSSION · boosted

  “• Confirmed that skewed delivery can take place on real-world ads for housing and employment opportunities by running a series of employment ads and housing ads with the same targeting parameters and bidding strategy.”

In the paper

“As the reader can see, these images are not discernible to the human eye.”

In the paper

“First, we can observe an ad delivery difference due to the destination URL itself; the base bodybuilding ad delivers to 48% men, while the base cosmetics ad delivers to 40% men.”

In the paper

“As shown in Figure 6A, we verify that the fraction of men in the delivery of the male ads is significantly higher than in female-centered and neutral ads, as well as higher in neutral ads than in female-centered ads.”

In the paper

“We can observe that ad delivery is, in fact, skewed, with the ads with stereotypically masculine images delivering to over 43% men and the ads with feminine images delivering to 39% men in the experiment targeting custom audiences as well as 58% and 44% respectively in the experiment targeting all U.S. users.”

In the paper

“We observe dramatic differences in ad delivery, even though the bidding strategy is the same for all ads, and each pair of ads target the same gender-agnostic audience.”

In the paper

“However, once we introduce the image into the ad, the delivery changes dramatically, returning to the level of skewed delivery discussed above (over 75% male for bodybuilding, and over 90% female for cosmetics).”

In the paper

“Interestingly, we also observe that the masculine invisible ads appear to be indistinguishable in the gender breakdown of their delivery from the masculine visible ads, and the feminine invisible ads appear to be indistinguishable in their delivery from the feminine visible ads.”

In the paper

“The images with the alpha channel are almost invisible to humans, but are still delivered in a skewed manner.”

In the paper

“we observe that all of our ads for lumberjacks deliver to an audience of primarily white and male users”

In the paper

“However, when running our experiments measuring race (and targeting specific DMAs), we observe that a fraction (∼10%) of our ads are delivered to audiences outside of our predefined DMAs, thus making it impossible for us to infer their race.”

In the paper

“Note that in experiments that involve measuring racial skew all ads target the same target audience.”

In the paper

“We calculate the lower and upper limits of the 99% confidence interval around this fraction using the method recommended by Agresti and Coull [9], defined in Equation 1:”

What the sources say

• ?Not checked — no full text of the sourcecites Agresti and Coull [9] 9. Alan Agresti and Brent A Coull. 1998. Approximate Is Better Than "exact" For Interval Estimation Of Binomial Proportions. The American Statistician 52, 2 (1998), 119-126. source

In the paper

“Facebook provides advertisers with an overview of how well-matched it believes an ad is with the target audience using a metric called relevance score, which ranges between 1 and 10.Facebook says [68]: Relevance score is calculated based on the positive and negative feedback we expect an ad to receive from its target audience.”

What the sources say

• ?Not checked — no full text of the sourcecites [68] 68. Piotr Sapiezynski, Wesley Zeng, Ronald E. Robertson, Alan Mislove, and Christo Wilson. 2019. Quantifying The Impact Of User Attention On Fair Group Representation In Ranked Lists. In Workshop on Fairness, Accountability, Transparency, Ethics, and Society on the Web. San Francisco, California, USA. source

In the paper

“For example, Lambrecht et al. ran a series of ads for STEM education and found they were consistently delivered more to men than to women, even though there are more female users on Facebook, and they are known to be more likely to click on ads and generate conversions [52].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [52] 52. Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna Gummadi, and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources Of Bias For Political Searches In Social Media. In ACM Conference on Computer Supported Cooperative Work. Portland, Oregon, USA.

In the paper

“We ensure that each racial group (white and Black) from a set of DMAs has a matching number of records of the other group in the other set of DMAs.”

In the paper

“For example, it is well-known that certain users on advertising platforms are more valuable to advertisers than others [48,55,65].”

What the sources say

• ?Not checked — no full text of the sourcecites [48] 48. Matthew Kay, Cynthia Matuszek, and Sean A. Munson. 2015. Unequal Representation And Gender Stereotypes In Image Search Results For Occupations. In Annual Conference of the ACM Special Interest Group on Computer Human Interaction. source
• ?Not checked — no full text of the sourcecites [55] 55. Mathias Lecuyer, Riley Spahn, Yannis Spiliopolous, Augustin Chaintreau, Roxana Geambasu, and Daniel Hsu. 2015. Sunlight: Fine-grained Targeting Detection At Scale With Statistical Confidence. In ACM Conference on Computer and Communications Security. source
• ?Not checked — no full text of the sourcecites [65] 65. Ronald E. Robertson, Shan Jiang, Kenneth Joseph, Lisa Friedland, David Lazer, and Christo Wilson. 2018. Auditing Partisan Audience Bias Within Google Search. In Annual Conference of the ACM Special Interest Group on Computer Human Interaction. source

In the paper

“Thus, it is clear that Facebook attempts to identify the users within an advertiser's selected audience who they believe would find the ad most useful (i.e., those who are most likely to result in an optimization event) and shows the ad preferentially to those users.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [4] 4. About Ad Delivery [n.d.]. About Ad Delivery. https://www.facebook.com/business/help/1000688343301256.

In the paper

“According to Facebook's documentation this represents the users who did not not list their gender.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Facebook's documentation

In the paper

“We carefully control for any time-of-day effects that might be present due to different user demographics using Facebook at different times of the day: for any given experiment, we run all ads at the same time to ensure that any such effects are experienced equally by all ads.”

In the paper

“Facebook goes on to say [68]: Put simply, the higher an ad's relevance score, the less it will cost to be delivered.This is because our ad delivery system is designed to show the right content to the right people, and a high relevance score is seen by the system as a positive signal.”

What the sources say

• ?Not checked — no full text of the sourcecites [68] 68. Piotr Sapiezynski, Wesley Zeng, Ronald E. Robertson, Alan Mislove, and Christo Wilson. 2019. Quantifying The Impact Of User Attention On Fair Group Representation In Ranked Lists. In Workshop on Fairness, Accountability, Transparency, Ethics, and Society on the Web. San Francisco, California, USA. source

In the paper

“Fourth, we find that the ad image is likely automatically classified by Facebook, and that this classification can skew delivery from the beginning of the ad's run.”

In the paper

“To this end, we use random PII-based custom audiences, where we randomly select U.S. Facebook users to be included in mutually-exclusive audiences.By doing so, we can ensure that our ads are only competing against each other in the cases where we wish them to.”

In the paper

“We run each of the ads at the same time and with the same bidding strategy and budget.”

In the paper

“Thus, we can use this methodology to test whether there is an automatic image classifier present by examining whether running different transparent white ads results in different delivery.”

In the paper

“The stronger effect we see when targeting all U.S. users may be due to the additional freedom that the ad delivery system has when choosing who to deliver to, as this is a significantly larger audience.”

In the paper

“To eliminate the impact that market effects can have on delivery in our following experiments, we ensure that all runs of a given experiment use the same bidding strategy and budget limit.”

In the paper

“In particular, they observed that their ads were predominantly shown to men even though women had consistently higher click through rates (CTRs).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Lambrecht et al. [52]

In the paper

“When running these experiments, we keep the ad creative and targeted audience constant, only changing the bidding strategy to give Facebook different daily limits (thus, any ad delivery differences can be attributed to the budget alone).”

In the paper

“We run these ads for 24 hours, using the objective of Traffic, all targeting the same audience with the same bidding strategy.”

In the paper

“Recall that the only difference between these ads are the ad creative and destination link; we (the advertiser) used the same bidding strategy and target audience, and ran all ads at the same time.”

In the paper

“Finally, we run the experiment twice, once targeting our random phone number custom audiences, and once targeting all users located in U.S.; we do so to verify that any effect we see is not a function of our particular target audience, and that it persists also when non-custom audiences are targeted.”

In the paper

“For all 25 of these ads, we hold the ad headline, text, and destination link constant, run them all at the same time, and use the same bidding strategy and target custom audiences in a way that each user is potentially exposed to up to three ads (one masculine image, one feminine image, and one blank image).”

In the paper

“All of the ads ran for 12 hours, using the objective of Traffic, all targeting the same North Carolina audiences and using the same bidding strategy.”

In the paper

“They then hypothesized that the higher CTRs led to women being more expensive to advertise to, meaning they were more likely to lose auctions for women when compared to auctions for men.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Lambrecht et al. [52]

In the paper

“Thus, we can conclude that the difference in ad delivery of our invisible male and female images is statistically significant, despite the fact that humans would not be able to perceive any differences in these ads.”

In the paper

“Note that we do not explicitly target either ad based on gender; the only targeting restrictions we stipulate are 18+ year old users in the U.S.”

In the paper

“Finally, we create a suite of ads that advertise a variety of housing opportunities, as discrimination in online housing ads has recently been a source of concern [32].”

What the sources say

• Not checked yetcites [32] 32. Facebook Engages in Housing Discrimination With Its Ad Practices, U.S. Says [n.d.]. Facebook Engages In Housing Discrimination With Its Ad Practices, U.S. Says. https://www.nytimes.com/2019/03/28/us/politics/ facebook-housing-discrimination.html.

In the paper

“the skew in delivery of employment and housing ads is potentially discriminatory in a legal sense.”

In the paper

“These effects can be invisible to, and might even create liability for, Facebook's advertisers.”

In the paper

“we use the Bonferroni correction [14], a statistical technique used to address the problem of making multiple comparisons.”

What the sources say

• ?Not checked — no full text of the sourcecites [14] 14. Robert J. Cabin and Randall J. Mitchell. 2000. To Bonferroni Or Not To Bonferroni: When And How Are The Questions. Bulletin of the Ecological Society of America 81, 3 (2000), 246-248. source

In the paper

“the skew we observe in the delivery of ads for cosmetics or bodybuilding might be interpreted as reinforcing gender stereotypes but is unlikely to have legal implications.”

In the paper

“Facebook banned the use of certain attributes for housing ads, but many other, un-banned, mechanisms exist for advertisers that achieve the same outcome[69].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [69] 69. Showing Relevance Scores for Ads on Facebook [n.d.]. Showing Relevance Scores For Ads On Facebook. https: //www.facebook.com/business/news/relevance-score.

In the paper

“Unfortunately, Facebook does not report the number of these users who match as we use multiple PII fields in the upload file[73].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [73] 73. Upturn Amicus Brief in Onuoha v. Facebook [n.d.]. Upturn Amicus Brief In Onuoha V. Facebook. https://www. courtlistener.com/recap/gov.uscourts.cand.304918/gov.uscourts.cand.304918.76.1.pdf.

In the paper

“Using incongruent images skews the delivery to a lesser degree, indicating that the image is not the only element of the ad that drives the skew in delivery.”

In the paper

“While ad targeting is facilitated by an advertising platform-but nominally controlled by advertisers-ad delivery is conducted and controlled by the advertising platform itself.”

In the paper

“Even absent unlawful behavior by advertisers, our research demonstrates that Facebook's own, independent actions during the delivery phase are crucial to determining how, when, and to whom ads are shown, and might produce unlawful outcomes.”

In the paper

“solely the fact that ads are not delivered to an evenly split audience does not indicate skew, as there might be differences in matching rates (what fraction of registered voters are active Facebook users) per ethnicity, or the groups could have different temporal usage patterns.”

In the paper

“Facebook agreed as part of a lawsuit settlement stemming from these issues to go further by banning age, gender, and certain kinds of location targeting-as well as some related attributes-for housing, employment, or credit ads[22].”

What the sources say

• Not checked yetcites [22] 22. Doing More to Protect Against Discrimination in Housing, Employment and Credit Advertising [n.d.]. Doing More To Protect Against Discrimination In Housing, Employment And Credit Advertising. https://newsroom.fb.com/news/ 2019/03/protecting-against-discrimination-in-ads/.

In the paper

“Only when we run two or more ads at the same time, targeting the same audience, and these ads are delivered with different proportions to white and Black users, do we claim we observe skew in delivery.”