CoChat
Sign Up
Cited by 21Ethics and Social Impacts of AIFace recognition and analysisSocial and Intergroup Psychology

Evidence for Hypodescent in Visual Semantic AI

Open access
Wolfe, Robert, Banaji, Mahzarin R., Caliskan, Aylin·2022·2022 ACM Conference on Fairness Accountability and Transparency
DOI 10.1145/3531146.3533185
Open this paper in CoChat — free accountOpen PDF

Abstract

We examine the state-of-the-art multimodal ”visual semantic” model CLIP (”Contrastive Language Image Pretraining”) for the rule of hypodescent, or one-drop rule, whereby multiracial people are more likely to be assigned a racial or ethnic label corresponding to a minority or disadvantaged racial or ethnic group than to the equivalent majority or advantaged group. A face morphing experiment grounded in psychological research demonstrating hypodescent indicates that, at the midway point of 1,000 series of morphed images, CLIP associates 69.7% of Black-White female images with a Black text label over a White text label, and similarly prefers Latina (75.8%) and Asian (89.1%) text labels at the midway point for Latina-White female and Asian-White female morphs, reflecting hypodescent. Additionally, assessment of the underlying cosine similarities in the model reveals that association with White is correlated with association with ”person,” with Pearson’s ρ as high as 0.82, p < 10− 90 over a 21,000-image morph series, indicating that a White person corresponds to the default representation of a person in CLIP. Finally, we show that the stereotype-congruent pleasantness association of an image correlates with association with the Black text label in CLIP, with Pearson’s ρ = 0.48, p < 10− 90 for 21,000 Black-White multiracial male images, and ρ = 0.41, p < 10− 90 for Black-White multiracial female images. CLIP is trained on English-language text gathered using data collected from an American website (Wikipedia), and our findings demonstrate that CLIP embeds the values of American racial hierarchy, reflecting the implicit and explicit beliefs that are present in human minds. We contextualize these findings within the history of and psychology of hypodescent. Overall, the data suggests that AI supervised using natural language will, unless checked, learn biases that reflect racial hierarchies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3531146.3533185

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multimodal generative AI models such as CLIP and the CLIP-filtered LAION dataset have harmful biases embedded in them.

    “Recent studies expose the harmful biases embedded in multimodal generative AI models such as CLIP (contrastive language-image pretraining; Wolfe et al. 2022) and the CLIP-filtered LAION dataset (Birhane et al. 2021), which are core components of generative AI models (e.g., Dall-E 2 or Stable Diffusion).”

    From Generative AI (2024) · cites this paper as Wolfe et al. 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 31

  1. Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. 2021. Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications. arXiv preprint arXiv:2108.02818 (2021). · arXiv
  2. Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963 (2021). · arXiv
  3. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016), 4349-4357.
  4. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accu- racy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77-91.
  5. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183-186.
  6. Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. 2020. Generative pretraining from pixels. In International Conference on Machine Learning. PMLR, 1691-1703.
  7. Jacob Cohen. 1992. Statistical power analysis. Current directions in psychological science 1, 3 (1992), 98-101.
  8. Emily Denton, Ben Hutchinson, Margaret Mitchell, Timnit Gebru, and Andrew Zaldivar. 2019. Image counterfactual sensitivity analysis for detecting unintended bias. arXiv preprint arXiv:1906.06439 (2019). · arXiv
  9. Jesse Dodge, Maarten Sap, Ana Marasovic, William Agnew, Gabriel Ilharco, Dirk Groeneveld, and Matt Gardner. 2021. Documenting the english colossal clean crawled corpus. arXiv preprint arXiv:2104.08758 (2021). · arXiv
  10. Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. 2021. Open-vocabulary Object Detection via Vision and Language Knowledge Distillation. arXiv preprint arXiv:2104.13921 2 (2021). · arXiv
  11. Nitzan Guetta, Asaf Shabtai, Inderjeet Singh, Satoru Momiyama, and Yuval Elovici. 2021. Dodging Attack Using Carefully Crafted Natural Makeup. arXiv preprint arXiv:2109.06467 (2021). · arXiv
  12. Jeff Heaton. 2020. Applications of deep neural networks. arXiv preprint arXiv:2009.05673 (2020). · arXiv
  13. Niharika Jain, Alberto Olmo, Sailik Sengupta, Lydia Manikonda, and Subbarao Kambhampati. 2020. Imperfect imaganation: Implications of gans exacerbating biases on facial data augmentation and snapchat selfie lenses. arXiv preprint arXiv:2001.09528 (2020). · arXiv
  14. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. 2020. Training Generative Adversarial Networks with Limited Data. In Proc. NeurIPS.
  15. Benedek Kurdi, Shayn Lozano, and Mahzarin R Banaji. 2017. Introducing the open affective standardized image set (OASIS). Behavior research methods 49, 2 (2017), 457-470.
  16. Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William T Freeman, Phillip Isola, Amir Globerson, Michal Irani, et al. 2021. Explaining in Style: Training a GAN to explain a classifier in StyleSpace. arXiv e-prints (2021), arXiv-2104.
  17. Zhiheng Li and Chenliang Xu. 2021. Discover the Unknown Biased Attribute of an Image Classifier. arXiv preprint arXiv:2104.14556 (2021). · arXiv
  18. Yuri Lin, Jean-Baptiste Michel, Erez Aiden Lieberman, Jon Orwant, Will Brock- man, and Slav Petrov. 2012. Syntactic annotations for the google books ngram corpus. In Proceedings of the ACL 2012 system demonstrations. 169-174.
  19. Debbie S Ma, Joshua Correll, and Bernd Wittenbrink. 2015. The Chicago face database: A free stimulus set of faces and norming data. Behavior research methods 47, 4 (2015), 1122-1135.
  20. Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. 2021. SLIP: Self- supervision meets Language-Image Pre-training. arXiv preprint arXiv:2112.12750 (2021). · arXiv
  21. Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2021. GLIDE: Towards Photoreal- istic Image Generation and Editing with Text-Guided Diffusion Models. arXiv preprint arXiv:2112.10741 (2021). · arXiv
  22. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 (2021). · arXiv
  23. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  24. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv preprint arXiv:2204.06125 (2022). · arXiv
  25. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. arXiv preprint arXiv:2102.12092 (2021). · arXiv
  26. Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2019. Contrastive Representation Distillation. In International Conference on Learning Representations.
  27. Autumn Toney-Wails and Aylin Caliskan. 2021. ValNorm Quantifies Seman- tics to Reveal Consistent Valence Biases Across Languages and Over Centuries. Empirical Methods in Natural Language Processing (EMNLP) (2021).
  28. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998-6008.
  29. Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern. 2019. Predictive in- equity in object detection. arXiv preprint arXiv:1902.11097 (2019). · arXiv
  30. Ellen D Wu. 2015. The color of success: Asian Americans and the origins of the model minority. Vol. 100. Princeton University Press.
  31. Michael A Zarate and Eliot R Smith. 1990. Person categorization and stereotyping. Social cognition 8, 2 (1990), 161-185.

Cited by

Sorted by citations

Related

Authors
Wolfe, R., Banaji, M. R., & Caliskan, A.
Affiliations
  • University of Washington (US)
  • Harvard University (US)
ORCID
Venue
2022 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2022-06-20
Pages
1293-1304
DOI
10.1145/3531146.3533185
OpenAlex
W4283171168
Cited by
19
References
45
Citations by year
2026: 3 · 2025: 5 · 2024: 4 · 2023: 6 · 2022: 1
FWCI
3.98
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
U.S. National Institute of Standards and Technology (NIST) (60NANB20D212)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
Keywords
hypodescent, one-drop rule, CLIP, racial bias in AI, face morphing, racial hierarchy, cosine similarity, implicit bias, natural language supervision

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review