Cited by 57Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Law, AI, and Intellectual Property

Investigating Gender and Racial Biases in DALL-E Mini Images

Open access
Cheong, Marc, Abedin, Ehsan, Ferreira, Marinus, Reimann, Ritsaart, Chalson, Shalom, Robinson, Pamela, Byrne, Joanne, Ruppanner, Leah +2 more·2024·ACM Journal on Responsible Computing

Abstract

Generative artificial intelligence systems based on transformers, including both text generators such as GPT-4 and image generators such as DALL-E 3, have recently entered the popular consciousness. These tools, while impressive, are liable to reproduce, exacerbate, and reinforce extant human social biases, such as gender and racial biases. In this article, we systematically review the extent to which DALL-E Mini suffers from this problem. In line with the Model Card published alongside DALL-E Mini by its creators, we find that the images it produces tend to represent dozens of different occupations as populated either solely by men (e.g., pilot, builder, plumber) or solely by women (e.g., hairdresser, receptionist, dietitian). In addition, the images DALL-E Mini produces tend to represent most occupations as populated primarily or solely by White people (e.g., farmer, painter, prison officer, software engineer) and very few by non-White people (e.g., pastor, rapper). These findings suggest that exciting new AI technologies should be critically scrutinized and perhaps regulated before they are unleashed on society.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3649883

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups, including people with disabilities.

    “Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 47 references couldn’t be matched to a known paper.

Research works printed but not linked: 8

  1. I Ajunwa, S A Friedler, C Scheidegger, and S Venkatasubramanian. 2016. Hiring by algorithm: predicting and preventing disparate impact. (2016). http://sorelle.friedler.net/papers/SSRN-id2746078.pdf
  2. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. [n. d.]. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. Adv. Neural Inf. Process. Syst. ([n. d.]). https://proceedings.neurips.cc/paper/2016/ile/ a486cd07e4ac3d270571622f4f316ec5-Paper.pdf
  3. Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M Ziegler, Jefrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. (May 2020). arXiv:2005.14165 [cs.CL] http://arxiv.org/abs/2005.14165 · arXiv
  4. Chris Dulhanty and Alexander Wong. 2019. Auditing ImageNet: Towards a Model-driven Framework for Annotating Demographic Attributes of Large-Scale Image Datasets. arXiv:1905.01347 [cs.LG] · arXiv
  5. Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. 2023. Stable Bias: Analyzing Societal Representations in Difusion Models. arXiv:2303.11408 [cs.CY] · arXiv
  6. Sheilla Njoto, Marc Cheong, Reeva Lederman, Aidan McLoughney, Leah Ruppanner, and Anthony Wirth. 2022. Gender Bias in AI Recruitment Systems: A Sociological-and Data Science-based Case Study. In Proceedings of the 2022 IEEE International Symposium on Technology and Society (ISTAS).
  7. Saiya Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press.
  8. Yi Chern Tan and L Elisa Celis. 2019. Assessing Social and Intersectional Biases in Contextualized Word Representations. arXiv (Nov. 2019). arXiv:1911.01485 [cs.CL] http://arxiv.org/abs/1911.01485 · arXiv

Cited by

Sorted by citations

Authors
Cheong, M., Abedin, E., Ferreira, M., Reimann, R., Chalson, S., Robinson, P., Byrne, J., Ruppanner, L., Alfano, M., & Klein, C.
Affiliations
  • The University of Melbourne (AU)
  • Macquarie University (AU)
  • Australian National University (AU)
Venue
ACM Journal on Responsible Computing
Published
2024-06-20
Volume
1
Issue
2
Pages
1-20
ISSN
2832-0565 (ISSN-L)
OpenAlex
W4392385810
Cited by
57
References
47
Citations by year
2026: 13 · 2025: 31 · 2024: 11 · 2023: 2
FWCI
22.90
Type
journal article
Language
en
Open access
Locations
  • ACM Journal on Responsible Computing (publishedVersion, cc-by)
  • PhilPapers (PhilPapers Foundation) (submittedVersion)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
gender bias, racial bias, generative image models, occupational stereotypes, AI regulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review