CoChat
Sign Up
Cited by 12Generative Adversarial Networks and Image SynthesisEthics and Social Impacts of AIMultimodal Machine Learning Applications

Towards Geographic Inclusion in the Evaluation of Text-to-Image Models

Open access
Hall, Melissa, Bell, Samuel J., Ross, Candace, Williams, Adina, Drozdzal, Michal, Soriano, Adriana Romero·2024·The 2024 ACM Conference on Fairness Accountability and Transparency
DOI 10.1145/3630106.3658927
Open this paper in CoChat — free account

Abstract

Rapid progress in text-to-image generative models coupled with their deployment for visual content creation has magnified the importance of thoroughly evaluating their performance and identifying potential biases. In pursuit of models that generate images that are realistic, diverse, visually appealing, and consistent with the given prompt, researchers and practitioners often turn to automated metrics to facilitate scalable and cost-effective performance profiling. However, commonly-used metrics often fail to account for the full diversity of human preference; often even in-depth human evaluations face challenges with subjectivity, especially as interpretations of evaluation criteria vary across regions and cultures. In this work, we conduct a large, cross-cultural study to study how much annotators in Africa, Europe, and Southeast Asia vary in their perception of geographic representation, visual appeal, and consistency in real and generated images from state-of-the art public APIs. We collect over 65,000 image annotations and 20 survey responses. We contrast human annotations with common automated metrics, finding that human preferences vary notably across geographic location and that current metrics do not fully account for this diversity. For example, annotators in different locations often disagree on whether exaggerated, stereotypical depictions of a region are considered geographically representative. In addition, the utility of automatic evaluations is dependent on assumptions about their set-up, such as the alignment of feature extractors with human perception of object similarity or the definition of “appeal” captured in reference datasets used to ground evaluations. We recommend steps for improved automatic and human evaluations. This includes collecting annotations from people located inside and outside the region of interest, instructing annotators on whether they should follow specific definitions of evaluation criteria or utilize their own interpretation, and reporting assumptions underlying automatic evaluations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3630106.3658927

Cites

14 of 17 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The identity and views of human annotators become embedded in the data, influencing subsequent system behavior.

    “In expressing 'their' perspective through data annotations, it important to recognize how the identity and views of human annotators become embedded in the data, influencing subsequent system behavior [51, 52, 90].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [52]

    The Introduction (p. 3) and Geographic Representation results (pp. 7–9) show that annotators’ locations and perspectives shape their judgments, supporting the point that those views are reflected in annotations. The paper compares annotations with evaluations of existing text-to-image models; it does not show that the annotations influence subsequent system behavior. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Hall et al. found that annotators who were not from the geographical region depicted in AI images tended to favor exaggerated, stereotypical images and overlooked other realistic, representative portrayals.

    “For example, Hall et al. [52] found that annotators, who were not from the geographical region depicted in AI images tended to favor exaggerated, stereotypical images and overlooked other realistic, representative portrayals.”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [52]

  • Where humans annotate data for AI image evaluation, their role commonly involves identifying objects or features in images, image comparisons, or gamified approaches.

    “Where humans annotate data for AI image evaluation, their role commonly involves: identifying objects or features in images [52]; image comparisons to indicate similarities or preferences across two or more samples (e.g., [52, 85]); or gamified approaches (e.g., describing artifacts characteristics for another annotator to guess [99]).”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [52]

  • Variation in how annotators subjectively interpret and follow concrete definitions has also been noted in other image generation data annotation work.

    “Variation in how annotators subjectively interpret and follow concrete definitions has also been noted in other image generation data annotation work [52].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [52]

  • In more community-centric data collection or AI evaluation work, active efforts are made by researchers to solicit broad participation or allow community members to self-select their participation, albeit with some limitations.

    “In more community-centric data collection or AI evaluation work, active efforts are made by researchers to solicit broad participation (e.g., across geographical regions [52, 68, 91] and through local research partners [89]), or allow community members to selfselect their participation, albeit with some limitations (e.g., adults, higher-education levels or topic expertise, English-language ability”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [52]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. [1][1] Idris Akinade, Jesujoba Alabi, David Adelani, Clement Odoje, and Dietrich Klakow. 2023. Epsilon kú mask: Integrating Yorùbá cultural greetings into machine translation. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) . Association for Computational Linguistics, Dubrovnik, Croatia, 1-7. https://doi.org/10.18653/v1/2023.c3nlp-1.1 - · DOI
  2. [5][5] Abhipsa Basu, R Venkatesh Babu, and Danish Pruthi. 2023. Inspecting the Geographical Representativeness of Images from Text-to-Image Models. arXiv preprint arXiv:2305.11080 (2023). - · arXiv
  3. [7][7] Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2014. Concreteness ratings for 40 thousand generally known English word lemmas. Behavior research methods 46, 3 (2014), 904-911. -
  4. [10][10] Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, Matthew Yu, Abhishek Kadian, Filip Radenovic, Dhruv Mahajan, Kunpeng Li, Yue Zhao, Vladan Petrovic, Mitesh Kumar Singh, Simran Motwani, Yi Wen, Yiwen Song, Roshan Sumbaly, Vignesh Ramanathan, Zijian He, Peter Vajda, and Devi Parikh. 2023. Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack. arXiv:2309.15807 [cs.CV] - · arXiv
  5. [11][11] Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2021. Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations. CoRR abs/2110.05719 (2021). arXiv:2110.05719 https://arxiv.org/abs/2110.05719 - · arXiv
  6. [12][12] Terrance De Vries, Ishan Misra, Changhan Wang, and Laurens Van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . 52-59. -
  7. [14][14] Sunipa Dev, Jaya Goyal, Dinesh Tewari, Shachi Dave, and Vinodkumar Prabhakaran. 2023. Building Socio-culturally Inclusive Stereotype Resources with Community Engagement. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track . https: //openreview.net/forum?id=uIj1jDc8k6 -
  8. [15][15] Esin Durmus, Karina Nyugen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2023. Towards Measuring the Representation of Subjective Global Opinions in Language Models. arXiv:arXiv:2306.16388 - · arXiv
  9. [17][17] Dan Friedman and Adji Bousso Dieng. 2022. The Vendi Score: A Diversity Evaluation Metric for Machine Learning. arXiv preprint arXiv:2210.02410 (2022). - · arXiv
  10. [18][18] Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. 2022. Make-a-scene: Scene-based text-to-image generation with human priors. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XV . Springer, 89-106. -
  11. [20][20] Priya Goyal, Adriana Romero Soriano, Caner Hazirbas, Levent Sagun, and Nicolas Usunier. 2022. Fairness Indicators for Systematic Assessments of Visual Feature Extractors. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22) . Association for Computing Machinery, New York, NY, USA, 70-88. -
  12. [21][21] Laura Gustafson, Megan Richards, Melissa Hall, Caner Hazirbas, Diane Bouchacourt, and Mark Ibrahim. 2023. Pinpointing Why Object Recognition Performance Degrades Across Income Levels and Geographies. arXiv:2304.05391 [cs.CV] - · arXiv
  13. [22][22] Melissa Hall, Candace Ross, Adina Williams, Nicolas Carion, Michal Drozdzal, and Adriana Romero Soriano. 2023. DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity. arXiv:2308.06198 [cs.CV] - · arXiv
  14. [23][23] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718 (2021). - · arXiv
  15. [24][24] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2018. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. arXiv:1706.08500 [cs.LG] - · arXiv
  16. [25][25] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Günter Klambauer, and Sepp Hochreiter. 2017. GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium. CoRR abs/1706.08500 (2017). arXiv:1706.08500 http://arxiv.org/abs/1706.08500 - · arXiv
  17. [27][27] Akshita Jha, Aida Davani, Chandan K. Reddy, Shachi Dave, Vinodkumar Prabhakaran, and Sunipa Dev. 2023. SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models. arXiv:arXiv:2305.11840 - · arXiv
  18. [28][28] Tuomas Kynkäänniemi, Tero Karras, Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2023. The Role of ImageNet Classes in Fréchet Inception Distance. arXiv:2203.06026 [cs.CV] - · arXiv
  19. [29][29] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. 2019. Improved Precision and Recall Metric for Assessing Generative Models. arXiv:1904.06991 [stat.ML] - · arXiv
  20. [31][31] Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Benita Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Li Fei-Fei, Jiajun Wu, Stefano Ermon, and Percy Liang. 2023. Holistic Evaluation of Text-To-Image Models. arXiv:2311.04287 [cs.CV] - · arXiv
  21. [32][32] Stanislav Morozov, Andrey Voynov, and Artem Babenko. 2021. On Self-Supervised Image Representations for {GAN} Evaluation. In International Conference on Learning Representations . https://openreview.net/forum?id=NeRdBeTionN -
  22. [34][34] Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. 2020. Reliable Fidelity and Diversity Metrics for Generative Models. CoRR abs/2002.09797 (2020). arXiv:2002.09797 https://arxiv.org/abs/2002.09797 - · arXiv
  23. [35][35] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2021. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. CoRR abs/2112.10741 (2021). arXiv:2112.10741 https: //arxiv.org/abs/2112.10741 - · arXiv
  24. [36][36] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. 2023. DINOv2: Learning Robust Visual Features without Supervision. arXiv:2304.07193 [cs.CV] - · arXiv
  25. [38][38] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. arXiv:2307.01952 [cs.CV] - · arXiv
  26. [39][39] Vinodkumar Prabhakaran, Rida Qadri, and Ben Hutchinson. 2022. Cultural Incongruencies in Artificial Intelligence. arXiv:arXiv:2211.13069 - · arXiv
  27. [42][42] Vikram V. Ramaswamy, Sing Yu Lin, Dora Zhao, Aaron B. Adcock, Laurens van der Maaten, Deepti Ghadiyaram, and Olga Russakovsky. 2023. Beyond web-scraping: Crowd-sourcing a geodiverse dataset. In arXiv preprint . -
  28. [43][43] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv:2204.06125 [cs.CV] - · arXiv
  29. [44][44] Suman V. Ravuri and Oriol Vinyals. 2019. Classification Accuracy Score for Conditional Generative Models. CoRR abs/1905.10887 (2019). arXiv:1905.10887 http://arxiv.org/abs/1905.10887 - · arXiv
  30. [45][45] Megan Richards, Polina Kirichenko, Diane Bouchacourt, and Mark Ibrahim. 2023. Does Progress On Object Recognition Benchmarks Improve Real-World Generalization? arXiv:arXiv:2307.13136 - · arXiv
  31. [46][46] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2021. High-Resolution Image Synthesis with Latent Diffusion Models. CoRR abs/2112.10752 (2021). arXiv:2112.10752 https://arxiv.org/abs/2112.10752 - · arXiv
  32. [47][47] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. 2022. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. arXiv:2205.11487 [cs.CV] - · arXiv
  33. [48][48] Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lučić, Olivier Bousquet, and Sylvain Gelly. 2018. Assessing Generative Models via Precision and Recall. In Advances in Neural Information Processing Systems (NeurIPS) . -
  34. [49][49] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. Advances in neural information processing systems 29 (2016). -
  35. [51][51] Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D Sculley. 2017. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. arXiv preprint arXiv:1711.08536 (2017). - · arXiv
  36. [52][52] Konstantin Shmelkov, Cordelia Schmid, and Karteek Alahari. 2018. How good is my GAN? CoRR abs/1807.09499 (2018). arXiv:1807.09499 http://arxiv.org/abs/1807.09499 - · arXiv
  37. [53][53] Mannat Singh, Laura Gustafson, Aaron Adcock, Vinicius de Freitas Reis, Bugra Gedik, Raj Prateek Kosaraju, Dhruv Mahajan, Ross Girshick, Piotr Dollár, and Laurens van der Maaten. 2022. Revisiting Weakly Supervised Pre-Training of Visual Perception Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 804-814. -
  38. [54][54] George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, J. Eric T. Taylor, and Gabriel Loaiza-Ganem. 2023. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. arXiv:2306.04675 [cs.LG] - · arXiv
  39. [55][55] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2015. Rethinking the Inception Architecture for Computer Vision. CoRR abs/1512.00567 (2015). arXiv:1512.00567 http://arxiv.org/abs/1512.00567 - · arXiv
  40. [57][57] Sharon Zhou, Mitchell L. Gordon, Ranjay Krishna, Austin Narcomey, Durim Morina, and Michael S. Bernstein. 2019. HYPE: Human eYe Perceptual Evaluation of Generative Models. CoRR abs/1904.01121 (2019). arXiv:1904.01121 http://arxiv.org/abs/1904.01121 · arXiv

Cited by

Sorted by citations

Related

Authors
Hall, M., Bell, S. J., Ross, C., Williams, A., Drozdzal, M., & Soriano, A. R.
Affiliations
  • Meta (United States) (US)
  • Meta (FAIR), France
  • Meta (FAIR), Canada
  • Meta (FAIR), Canada and Mila, Canada and McGill University, Canada
ORCID
Venue
The 2024 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2024-06-03
Pages
585-601
DOI
10.1145/3630106.3658927
OpenAlex
W4399365301
Cited by
11
References
14
Citations by year
2026: 8 · 2025: 3
FWCI
2.27
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
text-to-image models, geographical representation, cross-cultural assessment, visual appeal, image consistency, automatic evaluation metrics, stereotyping, regional variation, cross-cultural perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review