Cited by 783Ethics and Social Impacts of AINames, Identity, and Discrimination ResearchDiscrimination and Equality Law

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

Discrimination in online ad delivery

Sweeney, Latanya·2013·Communications of the ACM

DOI 10.1145/2447976.2447990

Open this paper in CoChat — free account

Abstract

Google ads, black names and white names, racial discrimination, and click advertising.

Integrity signals

Registry

Crossref · 10.1145/2447976.2447990

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination
Bertrand, M., & Mullainathan, S. (2004) · American Economic Review · doi:10.1257/0002828042002561 · Cited by 4,504
A sentimental education
Pang, B., & Lee, L. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218990 · Cited by 3,376
The social work dictionary
(2004) · Choice Reviews Online · doi:10.5860/choice.41-2555 · Cited by 1,030
The Causes and Consequences of Distinctively Black Names
Fryer, R. G., & Levitt, S. D. (2004) · The Quarterly Journal of Economics · doi:10.1162/0033553041502180 · Cited by 669
Discrimination in Online Ad Delivery: Google ads, black names and white names, racial discrimination, and click advertising
Sweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Self-citation
Uniform Guidelines on Employee Selection Procedures
Scheer, T. J. (2007) · Routledge eBooks · doi:10.1081/e-epap2-407 · Cited by 403
Equal Employment Opportunity Commission
(2012) · CQ Press eBooks · doi:10.4135/9781452241067.n3 · Cited by 275
Ex-Offenders Need Not Apply
Harris, P. M., & Keller, K. S. (2004) · Journal of Contemporary Criminal Justice · doi:10.1177/1043986204271678 · Cited by 154
The Causes and Consequences of Distinctively Black Names
Fryer, R. G., & Levitt, S. D. (2003) · National Bureau of Economic Research · doi:10.3386/w9938 · Cited by 55
Fairness Through Awareness
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2011) · arXiv · doi:10.48550/arxiv.1104.3913 · Cited by 55

Show all 10 references

Research works printed but not linked: 6

Central Hudson Gas & Electric Corp , v. Public Service Commission of New York. Supreme Court of the United States, 447 U.S. 557 , 1980 . Central Hudson Gas & Electric Corp, v. Public Service Commission of New York. Supreme Court of the United States, 447 U.S. 557, 1980.
Google . Google announces first quarter 2011 financial results ; http://investor.google.com/earnings/ 2011 /Q1_google_earnings.html. Google. Google announces first quarter 2011 financial results; http://investor.google.com/earnings/2011/Q1_google_earnings.html.
Panel on Methods for Assessing Discrimination , National Research Council. Measuring racial discrimination . National Academy Press , Washington, DC , 2004 . Panel on Methods for Assessing Discrimination, National Research Council. Measuring racial discrimination. National Academy Press, Washington, DC, 2004.
Sweeney , L. Discrimination in online ad delivery ( 2013 ). ( For de tails, see full technical report at http://ssrn.com/abstract=2208240. Data, including Web pages and ads, archived at http://foreverdata.org/onlineads). Sweeney, L. Discrimination in online ad delivery (2013). (For details, see full technical report at http://ssrn.com/abstract=2208240. Data, including Web pages and ads, archived at http://foreverdata.org/onlineads).
U.S. Commission on Civil Rights. Racism in America and how to combat it . Washington, DC , 1970 . U.S. Commission on Civil Rights. Racism in America and how to combat it. Washington, DC, 1970.
WebShot Command Line Server Edition. Version 1.9.1.1; http://www.websitescreenshots.com/. WebShot Command Line Server Edition. Version 1.9.1.1; http://www.websitescreenshots.com/.

Sorted by citations

Items 301–400 of 754

Weighing the benefits and risks of collecting race and ethnicity data in clinical settings for medical artificial intelligence
Fiske, A., Blacker, S., Geneviève, L. D., Willem, T., Fritzsche, M.-C., Buyx, A., Celi, L. A., & McLennan, S. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.01.003 · Cited by 21
Freedom and Constraint in Digital Environments: Implications for the Self
Talaifar, S., & Lowery, B. S. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221098036 · Cited by 21
“For example, Google is more likely to serve ads about incarceration in response to searches for names belonging to Black than White people (Sweeney, 2013) and ads for high-paying jobs to male users than female users (Datta et al., 2014).”
Background
FFTree: A flexible tree to handle multiple fairness criteria
Castelnovo, A., Cosentini, A., Malandri, L., Mercorio, F., & Mezzanzanica, M. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103099 · Cited by 17
On the Cruelty of Really Writing a History of Machine Learning
Plasek, A. (2016) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2016.43 · Cited by 17
Techniques to Elimenate Human Bias in Machine Learning
Sengupta, E., Garg, D., Choudhury, T., & Aggarwal, A. (2018) · International Conference on System Modeling & Advancement in Research Trends (SMART) · doi:10.1109/sysmart.2018.8746946 · Cited by 17
Digital Discrimination in Sharing Economy at the Base of the Pyramid
Attri, P. S., & Bapuji Hari, B. (2021) · Springer eBooks · doi:10.1007/978-981-16-2414-8_10 · Cited by 17
Is it Research or is it Spying? Thinking-Through Ethics in Big Data AI and Other Knowledge Sciences
Berendt, B., Büchler, M., & Rockwell, G. (2015) · Künstliche Intell. · doi:10.1007/s13218-015-0355-2 · Cited by 17
“Data’’ or [40] ‘‘Discrimination in Online Ads.”
Accessing the Impact of TikTok’s Algorithm on Regional Inequality in Health Information
Li, J., & Shi, W. (2024) · Health Communication · doi:10.1080/10410236.2024.2414882 · Cited by 17
“Much of the existing scholarship on algorithm audits has focused on demographic dimensions such as race (Sweeney, 2013) and gender (Datta et al., 2014).”
Investigating the lack of diversity in user behavior: The case of musical content on online platforms
Poulain, R., & Tarissan, F. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.102169 · Cited by 18
“Can algorithms really be objective and neutral [3]? What impact do they have on the users exposed to their output [4]? Do they shape and narrow our vision of what information is available online [5]? Do they generate any kind of discriminations when applied to large-scale infrastructures [6, 7]? These issues are all a…”
Background
Digital Economic Policy
Mariniello, M. (2022) · doi:10.1093/oso/9780198831471.001.0001 · Cited by 16
The Princeton Web Transparency and Accountability Project
Narayanan, A., & Reisman, D. (2017) · Studies in big data · doi:10.1007/978-3-319-54024-5_3 · Cited by 16
“Occasionally, researchers have rolled up their sleeves and conducted experiments manually in the absence of automated tools [70].”
Background
ISJ Editorial
Trauth, E. M., Joshi, K. D., & Yarger, L. K. (2018) · Information Systems Journal · doi:10.1111/isj.12201 · Cited by 16
“Sweeney (2013) shows that advertisements for a background check service were more likely to be displayed after a search for names that are traditionally associated with African Americans.”
Background
Patterns of Inclusion
Kelan, E. (2024) · Routledge eBooks · doi:10.4324/9781003427100 · Cited by 19
"Here the GPT made a choice, and every choice can be biased": How Students Critically Engage with LLMs through End-User Auditing Activity
Prabhudesai, S., Kasi, A. P., Mansingh, A., Antar, A. D., Shen, H., & Banović, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713714 · Cited by 27
The Ethics of Computational Social Science
Leslie, D. (2023) · Springer eBooks · doi:10.1007/978-3-031-16624-2_4 · Cited by 18
Author mentions in science news reveal widespread disparities across name-inferred ethnicities
Peng, H., Teplitskiy, M., & Jurgens, D. (2024) · Quantitative Science Studies · doi:10.1162/qss_a_00297 · Cited by 15
“…effective proxy for studying disparities at scale; and the literature suggests that self-identified ethnicity and the perceived one are highly correlated, and that humans can and do infer ethnicity from names fairly accurately (Gaddis, 2017a, 2017b; Sood & Laohaprapanon, 2018; Sweeney, 2013).”
Background
The ``Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biases
Das, D., Guha, S., Brubaker, J. R., & Semaan, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642669 · Cited by 27
“Traditionally, querying an algorithm with a wide range of inputs and statistically comparing the corresponding results has been one of the most effective ways for algorithmic audits [114, 169].”
Influential
Protecting Private Attributes in App Based Mobile User Profiling
Ullah, I., Boreli, R., Kanhere, S. S., Chawla, S., Ahanger, T. A., & Tariq, U. (2020) · IEEE Access · doi:10.1109/access.2020.3014424 · Cited by 16
“…have evaluated how users’ history can be tracked with browser ﬁngerprinting and cookie syncing [8], [9], the extent of web tracking [4], [40]–[42], user tracking on multiple devices [43], privacy implications using active and passive learning approaches [2], and how third party tracking…”
Background
Algorithmische Entscheidungssysteme revisited: Wie Maschinen gesellschaftliche Herrschaftsverhältnisse reproduzieren können
Prietl, B. (2019) · Feministische Studien · doi:10.1515/fs-2019-0029 · Cited by 15
An Algorithmic Framework to Control Bias in Bandit-based PersonalizationPreprint
Celis, L. E., Kapoor, S., Salehi, F., & Vishnoi, N. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08674 · Cited by 15
“Concerns have also been raised about gender and racial inequality in serving personalized advertising [10, 26, 12].”
Background
Envy-Free ClassificationPreprint
Balcan, M.-F., Dick, T., Noothigattu, R., & Procaccia, A. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.08700 · Cited by 15
“The study of fairness in machine learning is driven by an abundance of examples where learning algorithms were perceived as discriminating against protected groups (Sweeney, 2013; Datta et al., 2015).”
Background
Towards Accuracy-Fairness Paradox: Adversarial Example-based Data Augmentation for Visual DebiasingPreprint
Zhang, Y., & Sang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13632 · Cited by 15
“A biased model can lead to unintended social consequences such as in online advertisement, banking and criminal justice [1, 12, 24], e.”
Background
Artificial intelligence in international business: IB theory under augmented decision-making
Lindner, T., Puck, J. F., & Puhr, H. (2025) · Journal of World Business · doi:10.1016/j.jwb.2025.101676 · Cited by 21
What factors affect targeting and bids in online advertising?
Zeng, E., McAmis, R., Kohno, T., & Roesner, F. (2022) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3517745.3561460 · Cited by 14
“However, prior work has found evidence of targeting based on demographics factors [4, 12, 39].”
Background
Procedural Justice and Fairness in Automated Resume Parsers for Tech Hiring: Insights from Candidate Perspectives
Vaishampayan, S., Farzanehpour, S., & Brown, C. (2023) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc57772.2023.00019 · Cited by 17
“Previous work also demonstrates ML models can be biased based on gender [24], [29] and race/ethnicity [38].”
Background
Who's the Guinea Pig?
Jiang, S., Martin, J., & Wilson, C. (2019) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3287560.3287565 · Cited by 15
“This may occur due to exclusion (e.g., not showing ads for jobs [23] or housing to specific populations [3, 5]), or through association (e.g., showing criminal background check ads tied to the names of African American individuals [65]).”
Background
Censorship of Online Encyclopedias
Yang, E., & Roberts, M. E. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445916 · Cited by 20
“Because they are trained on human data, the algorithms themselves can replicate implicit and explicit human biases and aggravate discrimination [6, 8, 39].”
Background
Fair Learning in Markovian Environments.Preprint
Jabbari, S., Joseph, M., Kearns, M. J., Morgenstern, J., & Roth, A. (2016) · arXiv (Cornell University) · Cited by 14
ARTIFICIAL INTELLIGENCE (AI) BIAS IMPACTS: CLASSIFICATION FRAMEWORK FOR EFFECTIVE MITIGATION
(2023) · Issues in Information Systems · doi:10.48009/4_iis_2023_128 · Cited by 21
“…company examples include Google's Ad Settings page showing a preference for males over females for promotions connected with lucrative positions (Sweeney, 2013) and Facebook permitting its promoters to target advertisements as per various factors like race, religion, and gender, resulting in…”
Background
New Digital Divide Shaped by Algorithm? Evidence from Agent-Based Testing on Douyin’s Health-Related Video Recommendation
Shi, W., & Li, J. (2024) · Communication Research · doi:10.1177/00936502241262056 · Cited by 13
“For instance, evidence has indicated that advertising algorithms have displayed content in a seemingly discriminatory manner against women (Lambrecht & Tucker, 2019) and individuals with names that are indicative of a black identity (Sweeney, 2013).”
Do Black Fintechs Matter? The Long and Winding Road to Develop Inclusive Algorithms for Social Justice
Diniz, E. H., Sanches, B. H., Pozzebon, M., & Luvizan, S. S. (2024) · MIS Quarterly · doi:10.25300/misq/2024/18288 · Cited by 16
Explanation needs and ethical demands: unpacking the instrumental value of XAI
Alpsancar, S., Buhl, H. M., Matzner, T., & Scharlau, I. (2024) · AI and Ethics · doi:10.1007/s43681-024-00622-3 · Cited by 17
“Research in recent years has established discriminatory uses of AI concerning race and gender [ 136–138 ] or education [ 139].”
Data Governance and Human Rights: An Algorithm Discrimination Literature Review and Bibliometric Analysis
Wu, Y. (2023) · Journal of Humanities Arts and Social Science · doi:10.26855/jhass.2023.01.018 · Cited by 14
“Secondly, it could be because user behavior, such as clicking behavior, has driven machine learning algorithms to make these decisions based on specific names (Sweeney, 2013).”
Background
Fairness and Transparency in Crowdsourcing
Borromeo, R. M., Laurent, T., Toyama, M., & Amer-Yahia, S. (2017) · HAL · doi:10.5441/002/edbt.2017.46 · Cited by 13
“For example, Google’s advertising displays ads for high-income jobs to men much more often than it does to women; and ads for arrest records are most often associated to search queries for common African-American names [18].”
Background
On Fairness-Aware Learning for Non-discriminative Decision-Making
Zhang, W., Tang, X., & Wang, J. (2019) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2019.00157 · Cited by 13
“Increasing concerns have been raised from policy-makers, civil organizations, and researchers regarding the accountability and fairness of the employed decision-making models [17], [10], [28].”
Background
Quantifying 60 Years of Gender Bias in Biomedical Research with Word Embeddings
Rios, A., Joshi, R., & Shin, H. (2020) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2020.bionlp-1.1 · Cited by 15
“Likewise, ad-targeting algorithms may include characteristics of sexism and racism (Datta et al., 2015; Sweeney, 2013).”
Background
Risk mitigation in algorithmic accountability: The role of machine learning copies
Unceta, I., Nin, J., & Pujol, O. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241286 · Cited by 14
Causal Interventions for FairnessPreprint
Kusner, M. J., Russell, C., Loftus, J. R., & Silva, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02380 · Cited by 18
“For example, Google’s advertisement system was more likely to produce ads implying a person had been arrested when the search term was a name commonly associated with African Americans [32].”
Background
Algorithmic Discrimination in ServicePreprint
Ukanwa, K., & Rust, R. T. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3654943 · Cited by 12
DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision
Han, S., Lee, S., Wu, F., Kim, S., Wu, C., Wang, X., Xie, X., & Cha, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583480 · Cited by 12
“Machine learning techniques are being used in many mission-critical real-world Web applications, such as recommendation [1], hiring [25], and advertisement of profiles [41].”
Method
Explore the AI Anxiety Attitude of E-Commerce Employees
Chen, S., & Zhao, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2409470 · Cited by 12
Fairness in Search Systems
Yi, F., Singh, A., & Tao, Z. (2024) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000101 · Cited by 13
Dimensionen von Big Data: Eine politikwissenschaftliche Systematisierung
Ulbricht, L., Haunss, S., Hofmann, J., Klinger, U., Passoth, J.-H., Pentzold, C., Schneider, I., Straßheim, H., & Voß, J.-P. (2018) · Technikzukünfte, Wissenschaft und Gesellschaft · doi:10.1007/978-3-658-21665-8_3 · Cited by 13
Data Anonymization With Diversity Constraints
Milani, M., Huang, Y., & Chiang, F. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3131528 · Cited by 12
“Despite a strong need for algorithmic fairness and data diversity, such principles are rarely applied in practice [45].”
Background
The Independence of Fairness-Aware Classifiers
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2013) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2013.133 · Cited by 12
“Discovery of discriminative treatments [13], [14], [15], [16], [1], [17], [18], [9], learning non-discriminative estimators [3], [13], [7], and transforming data for non-discriminative analysis [11], [4] have been discussed.”
BackgroundMethodInfluential
Algorithms at Work: Empirical Diversity, Analytic Vocabularies, Design Implications
Wagenknecht, S., Lee, M., Lustig, C., O’Neill, J., & Zade, H. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818052.2855519 · Cited by 12
“However, algorithms can— intentionally or unintentionally—make discriminatory decisions that may affect users and workers [14, 15].”
Background
Managing Hypervisibility in the HIV Prevention Information‐Seeking Practices of Black Female College Students
Kvasny, L., & Payton, F. C. (2018) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24001 · Cited by 14
Thinking ahead to a world with quantum computers: the landscape of responsible research and innovation in quantum computing
Inglesant, P., Hartswood, M., & Jirotka, M. D. (2016) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 12
Who is more trustworthy, Alexa or mom?: Children’s selective trust in a digital age.
Murray, G. W. (2021) · Technology Mind and Behavior · doi:10.1037/tmb0000050 · Cited by 13
“Speechrecognition systems also are unabashed in providing search results that favor certain groups over others (Barocas & Selbst, 2016; Caliskan et al., 2017; Sweeney, 2013).”
Background
Fairness-aware machine learning: a perspectivePreprint
Žliobaitė, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.00754 · Cited by 12
“In recent years media and social research have pointed out plenty of anecdotal evidence that decision making by algorithms may unintentionally discriminate people [1,5–7, 16, 23].”
Background
Ethics of Artificial Intelligence in SurgeryPreprint
Rudzicz, F., & Saqur, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14302 · Cited by 17
“In contrast, unintended statistical biases caused by differences in sampling (or insufficient controls such as data regularization) may reflect and amplify existing cultural prejudices and inequalities [55].”
Background
Why resampling outperforms reweighting for correcting sampling bias with stochastic gradientsPreprint
An, J., Lexing Ying, B., & Zhu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13447 · Cited by 12
“Applying machine learning algorithms naively to biased training data can raise serious concerns and lead to controversial results (Sweeney, 2013; Kay et al., 2015; Menon et al., 2020).”
Background
Machine Behaviour (Originally Published 2019 by Springer Nature)
Rahwan, I., Cebrián, M., Obradovich, N., Bongard, J., Bonnefon, J., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2022) · Wiley eBooks · doi:10.1002/9781119815075.ch10 · Cited by 12
On the Societal Impact of Open Foundation Models
Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., Hopkins, A., Bankston, K., Biderman, S., Bogen, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07918 · Cited by 103
“The history of digital technology demonstrates that broader scrutiny, including by those belonging to marginalized groups that experience harm most acutely, reveals concerns missed by developers (Sweeney, 2013; Noble, 2018; Buolamwini & Gebru, 2018; Raji & Buolamwini, 2019).”
Cryptography, Trust and Privacy: It's Complicated
Balsa, E., Nissenbaum, H. F., & Park, S. (2022) · ACM Symposium on Computer Science and Law (CSLAW) · doi:10.1145/3511265.3550443 · Cited by 11
“Moreover, when search engines such as Google use search query data to personalize search results and target users with advertisements, users may be served hyperpersonalized queries and discriminated against, among other harms that stem from user profiling [47, 61].”
Method
Towards Just AI: Challenges and Solution Framework for Algorithmic Discrimination in Judicial System
Dou, L., & Dou, X. (2025) · International Journal of Digital Law and Governance · doi:10.1515/ijdlg-2024-0020 · Cited by 14
Iterative Effect-Size Bias in Ridehailing: Measuring Social Bias in Dynamic Pricing of 100 Million Rides.
Pandey, A., & Çalışkan, A. (2020) · Cited by 11
Bias in Deep Neural Networks in Land Use Characterization for International Development
Kim, D., López, G., Kiedanski, D., Maduako, I., Ríos, B., Descoins, A., Zurutuza, N., Arora, S., & Fabian, C. (2021) · Remote Sensing · doi:10.3390/rs13152908 · Cited by 12
“Efforts to document biases in algorithms and the ways in which this process plays out in practice have been made [15–18].”
Big Data in Practice
Broussard, M. (2015) · Digital Journalism · doi:10.1080/21670811.2015.1074863 · Cited by 14
“When a researcher develops a model based on patterns seen in data, the researcher may (intentionally or unintentionally) embed or perpetuate structural inequalities from the real world (O’Neil, forthcoming; Sweeney 2013).”
Background
Closed-Loop Opinion Formation
Spinelli, L., & Crovella, M. E. (2017) · ACM Web Science Conference (WebSci) · doi:10.1145/3091478.3091483 · Cited by 10
“Furthermore online price discrimination was exposed by [12] and online ad delivery discrimination was exposed by Sweeney in [25].”
Background
On Ensuring that Intelligent Machines Are Well-BehavedPreprint
Thomas, P. S., da Silva, B. C., Barto, A. G., & Brunskill, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05448 · Cited by 12
Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing ResearchPreprint
Havens, L., Terras, M., Bach, B., & Alex, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.05911 · Cited by 12
“Biased language causes representational harms (Vainapel et al., 2015; Sweeney, 2013), or the restriction of a person’s identity through the use of hyperbolic or simplistic language (Blodgett et al.”
Background
Evaluating and Mitigating Discrimination in Language Model Decisions
Tamkin, A., Askell, A., Lovitt, L., Durmus, E., Joseph, N., Kravec, S., Nguyen, K., Kaplan, J., & Ganguli, D. (2023) · arXiv · doi:10.48550/arxiv.2312.03689 · Cited by 11
“For example, Sweeney (2013) study discrimination in online ad delivery, finding that ads suggesting an arrest record are more likely to be shown with searches of Black-associated names.”
BackgroundMethod
Bias and Discrimination in Artificial Intelligence
Weyerer, J. C., & Langer, P. F. (2020) · Advances in e-business research series · doi:10.4018/978-1-7998-1879-3.ch011 · Cited by 12
Panel
Stoyanovich, J., Howe, B., Jagadish, H. V., & Miklau, G. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3229863.3240494 · Cited by 9
“In recent history, our technology has produced image labelers [19], search engines [23], and even criminal sentencing systems [4] that discriminate against and denigrate certain races as a byproduct of biased data.”
Background
The Human Side of Artificial Intelligence
Butkus, M. A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00239-9 · Cited by 10
“On the other hand, if we do not adopt a machine-learning model, we run the risk of a host of induced biases based on the nature of our cultural and personal preferences (Lloyd 2018; Sweeney 2013).”
Background
Practical AI Transparency: Revealing Datafication and Algorithmic Identities
Stefanija, A. P., & Pierson, J. (2020) · Journal of Digital Social Research · doi:10.33621/jdsr.v2i3.32 · Cited by 11
Principles for virtual health care to deliver real equity in diabetes
Kerr, D. E., & Sabharwal, A. (2021) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(21)00176-5 · Cited by 10
Digital alienation as the foundation of online privacy concerns
Dainow, B. (2016) · ACM SIGCAS Computers and Society · doi:10.1145/2874239.2874255 · Cited by 10
“This is achieved through personalization of content [56], such as the advertising [78] and news [81] to which people are exposed.”
Background
Quantifying Program BiasPreprint
Albarghouthi, A., D’Antoni, L., Drews, S., & Nori, A. V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05437 · Cited by 10
“Discrimination in black-box systems has been studied through the lens of statistical analysis [32, 33, 67].”
Background
Probabilistic Fair ClusteringPreprint
Esmaeili, S. A. H., Brubach, B., Tsepenekas, L., & Dickerson, J. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10916 · Cited by 10
“reviewers training data [14], advertising systems may discriminate based on race and inferred gender in harmful ways [43], recidivism risk assessment software may bias its risk assessment improperly by race [6], and healthcare resource allocation systems may be biased against a specific race [35].”
Background
On the Viability of Contextual Advertising as a Privacy-Preserving Alternative to Behavioral Advertising on the WebPreprint
Bleier, A. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3980001 · Cited by 10
Is it necessary? A framework for assessing the utility of A.I. in HRM practices
Sohani, S. S., Pandey, J., Varma, A., & Ray, P. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.104816 · Cited by 13
Representation of Music Creators on Wikipedia, Differences in Gender and Genre
Wang, A., Pappu, A., & Cramer, H. S. M. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18101 · Cited by 9
“Past studies show that biases often reﬂect existing societal inequities (Buolamwini and Gebru 2018; Sweeney 2013) and have highlighted the importance of representing identities well (Schlesinger, Edwards, and Grinter 2017).”
Background
Beijing’s central role in global artificial intelligence research
AlShebli, B., Cheng, E., Waniek, M., Jagannathan, R., Hernández‐Lagos, P., & Rahwan, T. (2022) · Scientific Reports · doi:10.1038/s41598-022-25714-0 · Cited by 12
Intersectional Thinking about PETs: A Study of Library Privacy
McDonald, N., Greenstadt, R., & Forte, A. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0064 · Cited by 11
“by police [5, 31, 81? ], the government [30], and online [48, 73, 86].”
Background
Popular Support for Balancing Equity and Efficiency in Resource Allocation: A Case Study in Online Advertising to Increase Welfare Program Awareness
Koenecke, A., Giannella, E., Willer, R., & Goel, S. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22163 · Cited by 11
"I'm Sorry, but I Can't Assist": Bias in Generative AI
Smith, J. M. (2024) · Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT) · doi:10.1145/3653666.3656065 · Cited by 14
“Previous work has identified extensive bias in domains ranging from online ad placement [31] to finance [12] and health care [24].”
Background
Leveraging Symptom Search Data to Understand Disparities in US Mental Health Care: Demographic Analysis of Search Engine Trace Data
Rochford, B., Pendse, S. R., Kumar, N., & Choudhury, M. D. (2022) · JMIR Mental Health · doi:10.2196/43253 · Cited by 9
The moral limits of the market: the case of consumer scoring data
Pham, A., & Castro, C. (2019) · Ethics and Information Technology · doi:10.1007/s10676-019-09500-7 · Cited by 14
Exploring Server-side Blocking of RegionsPreprint
Afroz, S., Tschantz, M. C., Sajid, S., Qazi, S. A., Javed, M., & Paxson, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11606 · Cited by 9
“Others have looked at ad personalization [21, 4, 53, 49, 34, 5, 33, 12, 51, 15, 50, 40, 8].”
Population-based Ensemble Learning with Tree Structures for Classification
Evans, B. (2019) · doi:10.26686/wgtn.17136296 · Cited by 8
Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
(2022) · doi:10.18653/v1/2022.gebnlp-1 · Cited by 9
FAIREDU: A multiple regression-based method for enhancing fairness in machine learning models for educational applications
Pham, N., Kha, M. B., Dai, T., Hùng, P. N., & Nguyen‐Duc, A. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126219 · Cited by 14
A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms
Harvey, E., Kizilcec, R. F., & Koenecke, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732137 · Cited by 11
“While initially used to audit prediction [22], classification [17], and recommendation [87] systems, this approach has more recently been adapted to audit generative AI models as well.”
Banking on a Revolution
Friedline, T. L. (2020) · doi:10.1093/oso/9780190944131.001.0001 · Cited by 8
Fairness in Online Ad Delivery
Baumann, J., Sapieżyński, P., Heitz, C., & Hannák, A. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658980 · Cited by 8
“…to suggest a criminal history when users search for individuals with Black-sounding names, even if those individuals have no such history [32, 43]; women may see fewer opportunity ads, especially if the advertisers operate on smaller budgets, because of the so-called competitive spillover…”
Background
Framing and the Malleability of Privacy Choices
Adjerid, I., Acquisti, A., & Loewenstein, G. F. (2014) · Cited by 8
Detecting Discrimination in a Black-Box Classifier
Alufaisan, Y., Kantarcıoğlu, M., & Zhou, Y. (2016) · IEEE International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic.2016.051 · Cited by 8
“Scoring function correction for naı̈ve Bayes and Decision Tree classifiers were used as a technique to prevent discrimination in [7] and [8].”
Method
On the Potential for Discrimination via Composition
Venkatadri, G., & Mislove, A. E. (2020) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3419394.3423641 · Cited by 8
“In addition, prior work demonstrated that particular real-world ads were delivered by Google’s platform to users in a skewed manner [11, 36], without inferring specific causes for this skewed outcome, while Datta et al.”
Background
Fairness-aware Model-agnostic Positive and Unlabeled Learning
Wu, Z., & He, J. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533225 · Cited by 10
“In applications such as credit approval [44], medical diagnosis [7], criminal risk assessment [38] and recommender systems [43, 47], fairness must be carefully taken into account to ensure the absence of discrimination against certain social groups (e.g., women, blacks).”
Background
Uncertainty and Inclusivity in Gender Bias Annotation: An Annotation Taxonomy and Annotated Datasets of British English Text
Havens, L., Alex, B., Bach, B., & Terras, M. (2022) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2022.gebnlp-1.4 · Cited by 9
“The need to mitigate bias in data has become urgent as evidence of harms from such data grows (Birhane and Prabhu, 2021; O’Neill et al., 2021; Perez, 2019; Noble, 2018; Vainapel et al., 2015; Sweeney, 2013).”
Background
Revisiting Group Fairness Metrics: The Effect of Networks
Mehrotra, A., Sachs, J., & Celis, L. E. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555100 · Cited by 8
“Numerous studies have found that a good aggregate outcome (across all individuals) can hide the disparities faced by individuals in particular groups [10, 15, 82].”
Background
Plausible Deniability in Web Search—From Detection to Assessment
Aonghusa, P. M., & Leith, D. J. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2769025 · Cited by 8
“In [14] an extensive review of adverts from Google and Reuters.”
The Invisible Power of Fairness. How Machine Learning Shapes Democracy
Beretta, E., Santangelo, A. D. M., Lepri, B., Vetrò, A., & Martin, J. C. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-18305-9_19 · Cited by 11
On the Direction of Discrimination: An Information-Theoretic Analysis of Disparate Impact in Machine Learning
Wang, H., Ustun, B., & Calmon, F. P. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437661 · Cited by 8
Fairness-Aware Learning with Prejudice Free Representations
Madhavan, R., & Wadhwa, M. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412150 · Cited by 8
Agency Laundering and Algorithmic Decision Systems
Rubel, A., Pham, A., & Castro, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15742-5_56 · Cited by 10
Fair Dimensionality Reduction and Iterative Rounding for SDPsPreprint
Morgenstern, J., Samadi, S., Singh, M., Tantipongpipat, U. T., & Vempala, S. (2019) · arXiv (Cornell University) · Cited by 8
Research Opportunities in Sociotechnical Interventions for Health Disparity ReductionPreprint
Siek, K. A., Veinot, T. C., & Mynatt, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01035 · Cited by 8
“While these biases can affect all aspects of our lives, from leisure [5] to criminal justice [6] to personal finances [7], they are all the more critical in the context of health and healthcare due to their significant personal and societal implications.”
Background

Authors: Sweeney, L.
Affiliations: Harvard University (US)
Corresponding author: Sweeney, Latanya
ORCID: Sweeney, Latanya
Venue: Communications of the ACM
Publisher: Association for Computing Machinery (ACM)
Published: 2013-05
Volume: 56
Issue: 5
Pages: 44-54
ISSN: 0001-0782 (ISSN-L)
1557-7317
DOI: 10.1145/2447976.2447990
OpenAlex: W2095932468
Cited by: 783
References: 10
Citations by year: 2026: 30 · 2025: 53 · 2024: 61 · 2023: 109 · 2022: 78 · 2021: 84 · 2020: 91 · 2019: 82 · 2018: 69 · 2017: 53 · 2016: 37 · 2015: 18 · 2014: 16 · 2013: 2
FWCI: 63.66
Type: journal article
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: Communications of the ACM (publishedVersion)
Funders: Division of Computer and Network Systems (CNS-1237235)
Google
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
Social Sciences › Law › Discrimination and Equality Law
Keywords: online advertising, racial discrimination, algorithmic discrimination, Google AdWords, algorithmic fairness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Discrimination in online ad delivery

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks