Measuring Representational Harms in Image Captioning
Open accessAbstract
Previous work has largely considered the fairness of image captioning systems through the underspecified lens of “bias.” In contrast, we present a set of techniques for measuring five types of representational harms, as well as the resulting measurements obtained for two of the most popular image captioning datasets using a state-of-the-art image captioning system. Our goal was not to audit this image captioning system, but rather to develop normatively grounded measurement techniques, in turn providing an opportunity to reflect on the many challenges involved. We propose multiple measurement techniques for each type of harm. We argue that by doing so, we are better able to capture the multi-faceted nature of each type of harm, in turn improving the (collective) validity of the resulting measurements. Throughout, we discuss the assumptions underlying our measurement approach and point out when they do not hold.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing research predominantly focuses on understanding, defining or evaluating representational 'harms' and often uses poor AI-generated images to elicit feedback from marginalized communities.
“Our approach to defining and evaluating 'good' representation differs from existing research that predominantly focuses on understanding, defining or evaluating representational 'harms' [24, 28, 38, 44, 67, 77, 91, 116] and often uses poor AI-generated images to elicit feedback from marginalized communities (e.g., [65, 77, 79]).”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [116]
The Introduction and §5.1 describe the paper’s aim as developing techniques to measure representational harms in image captioning, supporting its inclusion among research focused on harms. The methods use system-generated captions, while §5.1 proposes incorporating affected people’s lived experiences as future work. The paper does not support the claim that it used poor AI-generated images to elicit feedback from marginalized communities. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning
Lu, J., Xiong, C., Parikh, D., & Socher, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.345 · Cited by 1,724 · see citing works
Combining Local Context and WordNet Similarity for Word Sense Identification
Leacock, C. (1998) · The MIT Press eBooks · doi:10.7551/mitpress/7287.003.0018 · Cited by 1,674 · see citing works
Ethnicity and race : making identities in a changing world
Cornell, S., & Hartmann, D. (2007) · Cited by 1,120 · see citing works
VinVL: Revisiting Visual Representations in Vision-Language Models
Zhang, P., Li, X., Hu, X., Yang, J., Zhang, L., Wang, L., Choi, Y., & Gao, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00553 · Cited by 965 · see citing works
Murray, G., Renals, S. J., Carletta, J., & Moore, J. D. (2005) · Meeting of the Association for Computational Linguistics · Cited by 716 · see citing works
Honnibal, M., Montani, I., Honnibal, M., Peters, H., Samsonov, M., Geovedi, J., Regan, J., Orosz, G., Kristiansen, S. L., R., et al. (2018) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1212303 · Cited by 543 · see citing works
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Dai, B., Fidler, S., Urtasun, R., & Lin, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.323 · Cited by 483 · see citing works
Convolutional Image Captioning
Aneja, J., Deshpande, A., & Schwing, A. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00583 · Cited by 426 · see citing works
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404 · see citing works
(2017) · doi:10.1145/2998181.2998364 · Cited by 258 · see citing works
Yang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375709 · Cited by 254 · see citing works
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240 · see citing works
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
Shetty, R., Rohrbach, M., Hendricks, L. A., Fritz, M., & Schiele, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.445 · Cited by 231 · see citing works
Discriminability Objective for Training Descriptive Captions
Luo, R., Shakhnarovich, G., Cohen, S. D., & Price, B. L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00728 · Cited by 194 · see citing works
Understanding and predicting importance in images
Berg, A. C., Berg, T. L., Daumé, H., Dodge, J., Goyal, A., Han, X., Mensch, A., Mitchell, M., Sood, A., Stratos, K., et al. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248100 · Cited by 154 · see citing works
Learning to Evaluate Image Captioning
Cui, Y., Yang, G., Veit, A., Huang, X., & Belongie, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00608 · Cited by 140 · see citing works
Salisbury, E., Kamar, E. S., & Morris, M. R. (2017) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v5i1.13301 · Cited by 85 · see citing works
Stangl, A., Verma, N., Fleischmann, K. R., Morris, M. R., & Gurari, D. (2021) · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3471233 · Cited by 81 · see citing works
Girls, ladies, or women? the unconscious dynamics of language choice
Lerner, H. E. (1976) · Comprehensive Psychiatry · doi:10.1016/0010-440x(76)90003-1 · Cited by 61 · see citing works
Fairness in representation: quantifying stereotyping as a representational harm
Abbasi, M., Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2019) · Proceedings of the 2019 SIAM International Conference on Data Mining · doi:10.1137/1.9781611975673.90 · Cited by 44 · see citing works
Fairness in Proprietary Image Tagging Algorithms: A Cross-Platform Audit on People Images
Kyriakou, K., Barlas, P., Kleanthous, S., & Otterbacher, J. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3232 · Cited by 40 · see citing works
Describing Common Human Visual Actions in Images
Ronchi, M. R., & Perona, P. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.52 · Cited by 32 · see citing works
Social B(eye)as: Human and Machine Descriptions of People Images
Barlas, P., Kyriakou, K., Kleanthous, S., & Otterbacher, J. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3255 · Cited by 16 · see citing works
Enhancing Descriptive Image Captioning with Natural Language Inference
Shi, Z., Liu, H., & Zhu, X. (2021) · doi:10.18653/v1/2021.acl-short.36 · Cited by 15 · see citing works
Talking about other people: an endless range of possibilities
van Miltenburg, E., Elliott, D., & Vossen, P. T. J. M. (2018) · International Conference on Natural Language Generation · doi:10.18653/v1/w18-6550 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 37
- Mohsan Alvi, Andrew Zisserman, and Christoffer Nellaker. 2018. Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings. Workshop on Bias Estimation in Face Analytics at ECCV 2018 (2018).
- Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. SPICE: Semantic Propositional Image Caption Evaluation. European Conference on Computer Vision (ECCV) (2016).
- Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018).
- Pinar Barlas, Kyriakos Kyriakou, Olivia Guest, Styliani Kleanthous, and Jahna Otterbacher. 2020. To "See" is to Stereotype. ACM Conference on Computer- Supported Cooperative Work And Social Computing (CSCW) (2020).
- Pinar Barlas, Kyriakos Kyriakou, Styliani Kleanthous, and Jahna Otterbacher. 2021. Person, Human, Neither: The Dehumanization Potential of Automated Image Tagging. AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) (2021).
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The Problem With Bias: Allocative Versus Representational Harms in Machine Learn- ing. In Proceedings of SIGCIS. Philadelphia, PA.
- Shruti Bhargava and David Forsyth. 2019. Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models. arXiv:1912.00578 (2019). · arXiv
- Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O'Reilly Media Inc. (2009).
- Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny, pornography, and malignant stereotypes. arXiv:2110.01963 (2021). · arXiv
- Su Lin Blodgett, Solon Barocas, Hal Daume III, and Hanna Wallach. 2020. Lan- guage (Technology) is Power: A Critical Survey of "Bias" in NLP. Association for Computational Linguistics (ACL) (2020).
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accu- racy Disparities in Commercial Gender Classification. Conference on Fairness, Accountability and Transparency (FAccT) (2018).
- Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325 (2015). · arXiv
- Terrance DeVries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does Object Recognition Work for Everyone? Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2019).
- Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus. Conference on Empirical Methods in Natural Language Processing (EMNLP) (2021).
- Xiaotian Han, Jianwei Yang, Houdong Hu, Lei Zhang, Jianfeng Gao, and Pengchuan Zhang. 2021. Image Scene Graph Generation (SGG) Benchmark. arXiv:2107.12604 [cs.CV] · arXiv
- Jared Katzman, Solon Barocas, Su Lin Blodgett, Kristen Laird, Morgan Klaus Scheuerman, and Hanna Wallach. 2021. Representational Harms in Image Tagging. Beyond Fair Computer Vision Workshop at CVPR 2021 (2021). https: //drive.google.com/file/d/1oJp8CqNpYEsOlO8cwv4cTnHGbOjWxEZ-/view
- Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. 2016. Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. https://arxiv.org/abs/1602.07332 · arXiv
- Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao. 2020. Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. European Conference on Computer Vision (ECCV) (2020).
- Chenxi Liu, Junhua Mao, Fei Sha, and Alan Yuille. 2017. Attention Correctness in Neural Image Captioning. AAAI Conference on Artificial Intelligence (AAAI-17) (2017).
- Kelly Mack, Edward Cutrell, Bongshin Lee, and Meredith Ringel Morris. 2021. Designing Tools for High-Quality Alt Text Authoring. International ACM SIGAC- CESS Conference on Computers and Accessibility (ASSETS) (2021).
- Haley MacLeod, Cynthia L. Bennett, Meredith Ringel Morris, and Edward Cutrell. 2017. Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images. ACM Conference on Human Factors in Computing Systems (CHI) (2017).
- Jahna Otterbacher, Pınar Barlas, Styliani Kleanthous, and Kyriakos Kyriakou. 2019. How Do We Talk about Other People? Group (Un)Fairness in Natural Language Image Descriptions. The Seventh AAAI Conference on Human Computation and Crowdsourcing (HCOMP-19) (2019).
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. Association for Compu- tational Linguistics (ACL) (2002).
- Morgan Klaus Scheuerman, Aaron Jiang, Katta Spiel, and Jed R. Brubaker. 2021. Revisiting Gendered Web Forms: An Evaluation of Gender Inputs with (Non- )Binary People. ACM Conference on Human Factors in Computing Systems (CHI) (2021).
- Morgan Klaus Scheuerman, Kandrea Wade, Caitlin Lustig, and Jed R. Brubaker. 2020. How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis. ACM Conference on Human Factors in Computing Systems (CHI) (2020).
- Gun R. Semin and Klaus Fiedler. 1988. The cognitive functions of linguistic categories in describing persons: Social cognition and language. Journal of Personality and Social Psychology (1988).
- Abigale Stangl, Meredith Ringel Morris, and Danna Gurari. 2020. "Person, Shoes, Tree. Is the Person Naked?" What People with Vision Impairments Want in Image Descriptions. ACM Conference on Human Factors in Computing Systems (CHI) (2020).
- Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. 2020. Long-Tailed Classifi- cation by Keeping the Good and Removing the Bad Momentum Causal Effect. Conference on Neural Information Processing Systems (NeurIPS) (2020).
- Ruixiang Tang, Mengnan Du, Yuening Li, Zirui Liu, and Xia Hu. 2020. Mitigating Gender Bias in Captioning Systems. arXiv:2006.08315 (2020). · arXiv
- Emiel van Miltenburg. 2016. Stereotyping and Bias in the Flickr30K Dataset. Proceedings of the Workshop on Multimodal Corpora (MMC) (2016).
- Emiel van Miltenburg, Desmond Elliott, and Piek Vossen. 2018. Measuring the Diversity of Automatic Image Descriptions. International Conference on Computational Linguistics (2018).
- Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and Tell: A Neural Image Caption Generator. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2015).
- Angelina Wang, Arvind Narayanan, and Olga Russakovsky. 2020. REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets. European Conference on Computer Vision (ECCV) (2020).
- Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. 2019. Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations. International Conference on Computer Vision (ICCV) (2019).
- Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern. 2019. Predictive In- equity in Object Detection. arXiv:1902.11097 (2019). · arXiv
- Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. 2016. Image Captioning with Semantic Attention. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2016).
- Dora Zhao, Angelina Wang, and Olga Russakovsky. 2021. Understanding and Evaluating Racial Biases in Image Captioning. International Conference on Com- puter Vision (ICCV) (2021).
Cited by
Sorted by citations
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279 · see citing works
DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models
Cho, J., Zala, A., & Bansal, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00283 · Cited by 199 · see citing works
Zhang, R., Li, H., Meng, H., Zhan, J., Gan, H., & Lee, Y.-C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713429 · Cited by 123 · see citing works
“Unlike general-purpose AI chatbots that are often task-oriented, such as ChatGPT and Amazon Alexa, AI companions prioritize fostering long-term, emotional connections that mimic human relationships through advanced personalization, memory, and adaptive interactions.”
AI’s Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia
Qadri, R., Shelby, R., Bennett, C. L., & Denton, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594016 · Cited by 85 · see citing works
Deep image captioning: A review of methods, trends and future challenges
Xu, L., Tang, Q., Lv, J., Zheng, B., Zeng, X., & Li, W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126287 · Cited by 78 · see citing works
“They only care to show us the wheelchair”: disability representation in text-to-image AI models
Mack, K. A., Qadri, R., Denton, R., Kane, S. K., & Bennett, C. L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642166 · Cited by 67 · see citing works
Deng, W. H., Yildirim, N., Chang, M., Eslami, M., Holstein, K., & Madaio, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594037 · Cited by 57 · see citing works
Uncurated Image-Text Datasets: Shedding Light on Demographic Bias
García, N., Hirota, Y., Wu, Y., & Nakashima, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00672 · Cited by 45 · see citing works
Berman, G., Goyal, N., & Madaio, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642398 · Cited by 45 · see citing works
FairGRAPE: Fairness-Aware GRAdient Pruning mEthod for Face Attribute Classification
Lin, X., Kim, S., & Joo, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_24 · Cited by 35 · see citing works
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging
Katzman, J., Wang, A., Scheuerman, M., Blodgett, S. L., Laird, K., Wallach, H., & Barocas, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26670 · Cited by 32 · see citing works
Self-citationAwumey, E., Das, S., & Forlizzi, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658945 · Cited by 29 · see citing works
Overwriting Pretrained Bias with Finetuning Data
Wang, A., & Russakovsky, O. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00366 · Cited by 24 · see citing works
Self-citationBeyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation
Chien, J., & Danks, D. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658946 · Cited by 22 · see citing works
“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”
MethodBlack, E., Naidu, R. C. A., Ghani, R., Rodolfa, K. T., Ho, D. E., & Heidari, H. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623259 · Cited by 19 · see citing works
Conceptualizing Algorithmic Stigmatization
Andalibi, N., Pyle, C., Barta, K., Xian, L., Jacobs, A. Z., & Ackerman, M. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580970 · Cited by 18 · see citing works
Benchmark suites instead of leaderboards for evaluating AI fairness
Wang, A., Hertzmann, A., & Russakovsky, O. (2024) · Patterns · doi:10.1016/j.patter.2024.101080 · Cited by 17 · see citing works
Self-citationChallenging Fairness: A Comprehensive Exploration of Bias in LLM-Based Recommendations
Sakib, S. K., & Das, A. B. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825082 · Cited by 17 · see citing works
Stangl, A., Sadjo, E., Emami-Naeini, P., Wang, Y., Gurari, D., & Findlater, L. (2023) · 20th International Web for All Conference · doi:10.1145/3587281.3587296 · Cited by 15 · see citing works
Understanding and evaluating harms of AI-generated image captions in political images
Sarhan, H., & Hegelich, S. (2023) · Frontiers in Political Science · doi:10.3389/fpos.2023.1245684 · Cited by 14 · see citing works
A Framework for Socio-Developmental Ethics in Educational AI
Tuomi, I. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.752 · Cited by 14 · see citing works
AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development
Saxena, D., Jung, J.-Y., Forlizzi, J., Holstein, K., & Zimmerman, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714098 · Cited by 15 · see citing works
Building Solidarity Amid Hostility: Experiences of Fat People in Online Communities
Payne, B. H., Taylor, J., Spiel, K., & Fiesler, C. L. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701186 · Cited by 16 · see citing works
An Epistemic Lens on Algorithmic Fairness
Edenberg, E., & Wood, A. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623248 · Cited by 7 · see citing works
SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generation
Liu, Z., Schaldenbrand, P., Okogwu, B.-C., Peng, W., Peng, W., Yun, Y., Hundt, A., Kim, J., & Oh, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01029 · Cited by 12 · see citing works
Gebru, T., & Denton, E. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000102 · Cited by 7 · see citing works
VLSlice: Interactive Vision-and-Language Slice Discovery
Slyman, E., Kahng, M., & Lee, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01403 · Cited by 5 · see citing works
Slyman, E., Lee, S., Cohen, S. D., & Kafle, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01319 · Cited by 5 · see citing works
AI’s Regimes of Representation: A Community-Centered Study of Text-to-Image Models in South Asia
Qadri, R., Shelby, R. M., Bennett, C. L., & Denton, E. (2024) · doi:10.21428/2c646de5.2b73d5c2 · Cited by 4 · see citing works
‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Alenichev, A., Shaffer, J. D., Kingori, P., Grietens, K. P., Muldoon, J., & Rocher, L. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02685-0 · Cited by 3 · see citing works
“…for example, how an image of African people exhibited in a human zoo is algorithmically processed and labeled as ‘savages’ (e.g., Caption 1.1), drawing, for instance, on the work of (Wang et al. 2022a, b), who developed measurement techniques of observable properties in image captions and images.”
InfluentialBennett, C. L., Kane, S. K., & Harrington, C. N. (2025) · Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3757887.3763012 · Cited by 3 · see citing works
Explaining CLIP's Performance Disparities on Data from Blind/Low Vision Users
Massiceti, D., Longden, C., Słowik, A., Wills, S., Grayson, M., & Morrison, C. P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01157 · Cited by 12 · see citing works
ToViLaG: Your Visual-Language Generative Model is Also An Evildoer
Wang, X., Yi, X., Jiang, H., Zhou, S., Wei, Z., & Xie, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.213 · Cited by 8 · see citing works
Countering Privacy NihilismPreprint
Engelmann, S., & Nissenbaum, H. F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5173215 · Cited by 2 · see citing works
Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation
Thieme, A., Faia Marques, R., Grayson, M., Balachandar, S., Tyler Cassidy, C., Choksi, M. Z., Longden, C., Huda, R. S., Ileve Kalovwe, N., Mallon, C., et al. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790768 · Cited by 1 · see citing works
“Researchers have previously problematized anonymous crowdsourced data annotation approaches [31, 53, 65] as being unspecific about who’s worldviews and beliefs become manifest in the data [11, 17, 51, 78, 90, 91].”
Sarhan, H., Shahrezaye, M., & Hegelich, S. (2025) · AI and Ethics · doi:10.1007/s43681-025-00773-x · Cited by 1 · see citing works
Evaluating Metrics for Impact QuantificationPreprint
Jenkins, R., & Nericcio, L. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4607297
41Inclusive Leadership and the Representational Issues of Human-Like Robots
Mamak, K. (2025) · De Gruyter eBooks · doi:10.1515/9783111662145-003
Nericcio, L., & Jenkins, R. (2026) · AI and Ethics · doi:10.1007/s43681-026-01118-y
The Problem of Representational Harm in Materials Dataset Construction
Brown, G., Taylor, M., Wilson, S., & Harris, O. (2025) · Journal of Artificial Intelligence for Materials Science · doi:10.68159/s290614688
- Authors
- Wang, A., Barocas, S., Laird, K., & Wallach, H.
- Affiliations
- Princeton University (US)
- Microsoft (United States) (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 324-335
- OpenAlex
- W4283028305
- Cited by
- 40
- References
- 29
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 12 · 2023: 17 · 2022: 1
- FWCI
- 9.44
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2039656)
- Microsoft Research
- Microsoft
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- image captioning, representational harm, algorithmic fairness, bias measurement, fairness metrics, measurement validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
