Cited by 21Generative Adversarial Networks and Image SynthesisEthics and Social Impacts of AIMultimodal Machine Learning Applications

Inspecting the Geographical Representativeness of Images from Text-to-Image Models

Open access
Basu, Abhipsa, Babu, R. Venkatesh, Pruthi, Danish·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

Abstract

Recent progress in generative models has resulted in models that produce both realistic as well as relevant images for most textual inputs. These models are being used to generate millions of images everyday, and hold the potential to drastically impact areas such as generative art, digital marketing and data augmentation. Given their outsized impact, it is important to ensure that the generated content reflects the artifacts and surroundings across the globe, rather than over-representing certain parts of the world. In this paper, we measure the geographical representativeness of common nouns (e.g., a house) generated through DALL•E 2 and Stable Diffusion models using a crowdsourced study comprising 540 participants across 27 countries. For deliberately underspecified inputs without country names, the generated images most reflect the surroundings of the United States followed by India, and the top generations rarely reflect surroundings from all other countries (average score less than 3 out of 5). Specifying the country names in the input increases the representativeness by 1.44 points on average on a 5 − point Likert scale for DALL•E 2 and 0.75 for Stable Diffusion, however, the overall scores for many countries still remain low, highlighting the need for future models to be more geographically inclusive. Lastly, we examine the feasibility of quantifying the geographical representativeness of generated images without conducting user studies.1

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00474

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The most dominant approach to gathering human annotations is to 'crowd-source' either anonymous people or community members as 'data workers', who are given simple, tokenistic tasks with little or no context to the data.

    “The most dominant approach to gathering human annotations is to 'crowd-source' [7, 53, 65] either anonymous people or community members as 'data workers', who are given simple, tokenistic tasks 'that even inexperienced annotators can finish in an instance' [31, 85], with little or no context to the data.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 63 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. [8][8] Jaemin Cho, Abhay Zala, and Mohit Bansal. Dalleval: Probing the reasoning skills and social biases of text-to-image generative transformers. arXiv preprint arXiv:2202.04053 , 2022. 2, 7, 10 - · arXiv
  2. [10][10] Alexey Dosovitskiy, Lucas Beyer, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020. 8 - · arXiv
  3. [23][23] Vikram V Ramaswamy, Sing Yu Lin, et al. Beyond webscraping: Crowd-sourcing a geographically diverse image dataset. arXiv preprint arXiv:2301.02560 , 2023. 9 - · arXiv
  4. [25][25] Aditya Ramesh, Mikhail Pavlov, et al. Zero-shot text-toimage generation. In ICML , pages 8821-8831. PMLR, 2021. 3, 9 -
  5. [38][38] Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. The biased artist: Exploiting cultural biases via homoglyphs in text-guided image generation models. arXiv preprint arXiv:2209.08891 , 2022. 7 - · arXiv
  6. [40][40] Ming Tao, Hao Tang, et al. Df-gan: Deep fusion generative adversarial networks for text-to-image synthesis. arXiv preprint arXiv:2008.05865 , 2020. 9 - · arXiv

Cited by

Sorted by citations

Authors
Basu, A., Babu, R. V., & Pruthi, D.
Affiliations
Indian Institute of Science Bangalore (IN)
Venue
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2023-10-01
Pages
5113-5124
OpenAlex
W4390873263
Cited by
21
References
63
Citations by year
2026: 4 · 2025: 11 · 2024: 5 · 2023: 1
FWCI
2.25
Type
conference paper
Language
en
Open access
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
text-to-image models, DALL-E 2, Stable Diffusion, crowdsourced evaluation, cultural bias in AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review