Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation
Open accessAbstract
Media representations of people with disabilities profoundly influence societal perceptions, yet have historically been absent, stereotyped, or inaccurate. As AI-generated visual media becomes increasingly prevalent, there is a critical opportunity to address these misrepresentations. Responding to the lack of collectively negotiated representation standards, this paper presents our human-centric approach to engaging disability communities meaningfully in AI data practices. Over three months, we worked closely with three disability organizations across the Global North and South to develop the Community Library Creator that introduces design scaffolds to support communities in defining ‘good’ representation and curating community-centric AI datasets; laying the foundations for community-specific evaluation metrics and future model adaptations. We contribute qualitative insights into the complexities of community-led data curation; discuss the value and practical challenges of intersecting human insights with AI requirements; and reflect on human-centered AI approaches that empower communities to share their perspectives and actively shape AI data practices.
Verified 2026-10-25 · 1 reference not checked yet
AI snapshot
Written from the paper’s full text.
TL;DR
This paper presents a three-month, community-led approach with three disability advocacy organizations to define “good” disability representation and curate AI-interpretable image datasets via a prototype called the Community Library Creator. It contributes qualitative insights into the complexities of community-led data curation and the tensions between human representation goals and technical AI requirements.
Key findings
- Across three organizations—representing people with vision impairment in Kenya, people of short stature in Kenya, and people with dwarfism in the USA—communities defined representation in individualized, dynamic, and negotiated ways; themes included work, diversity, education, family, celebration, and inclusion, with each organization interpreting shared themes differently.
- The project produced a 400-image Community Library per organization, with image annotations and auto-generated prompts; all organizations reported being “very” satisfied (4/5) with their community’s representation, though they wanted more image quality and diversity (e.g., more children, elderly, family/groups, rarer disability types).
- Thematic balance was difficult: the Global North dwarfism community chose three rather than five main themes, resulting in 23–46 images in each of four sub-themes, while the vision impairment community had sub-themes with fewer images (e.g., digital economy 7, farming 13, competition/tournaments 13).
- In AI evaluation, project leads rated generated images on a 1–5 scale; for each community, five prompts from each of ∼20 sub-themes yielded 100 prompts across three image generation models, and after filtering offensive content, ∼300 images per community were rated, with judgments combining correctness, realism, respectfulness, authenticity, and assistive-technology accuracy.
Methods
- Qualitative, participatory design over three months with three disability advocacy organizations as proxies for bounded communities; project leads committed three days per week to webinars, interviews, platform use, and self-organized community outreach.
- The Community Library Creator prototype supported three phases: defining positive representation (Photo Pinboard, Magazine, community outreach), curating the Community Library (400 images with annotations and GPT-4o-generated image-level prompts), and evaluating AI (rating generated images 1–5).
- Data included recorded/transcribed meetings, system entries, post-study questionnaires, and shared materials; analysis used qualitative content analysis with data immersion, deductive and inductive coding, and interpretive summaries; the study was IRB approved (Reference ID: 11019).
Limitations
- Stated: the work is grounded in three disability organizations, so future work is needed to examine generalization across more diverse organizations and to assess whether communities could navigate the platform and process independently.
- Stated: open questions remain about how best to use the datasets in a rapidly evolving model landscape, including data scaling without compromising authenticity; the model-agnostic approach cannot eliminate model-specific errors, and future community-specific evaluator models would be constrained to the three image generation models used.
- Evident from the text: only three project leads (P1–P3) served as primary participants; outreach had absent groups (e.g., rural members, younger children/students, elderly members, LGBTQ sub-community); annotation inconsistencies occurred (over-annotation of general items, overlooked relevant details); and some sub-themes had fewer images than the proposed balanced structure.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
97 of 178 cited sources checked · 52 supported · 0 flagged · accuracy not yet measured
- Premisecitation§ 1 Introduction · p. 2
Media portrayals of people with disabilities have historically been absent, stereotyped, harmful or inaccurate rather than reflecting authentic, lived experiences.
“Yet, media portrayals of PwD have historically been absent, stereotyped, harmful or inaccurate rather than reflecting authentic, lived experiences [34, 49, 88, 114, 117].”
Quote in the paperSubstantial claim1 of 5 sources checked- cites [34] [34] Katie Ellis, Gerard Goggin, Beth A. Haller, and Rosemary Curtis (Eds.). 2020. The Routledge Companion to Disability and Media . Routledge, New York. https: //doi.org/10.4324/9781315716008 - source
- cites [49] [49] Tina Goethals, Dimitri Mortelmans, Hilde Van den Bulck, Willem Van den Heurck, and Geert Van Hove and. 2022. I am not your metaphor: frames and counter-frames in the representation of disability. Disability & Society 37, 5 (2022), 746-764. arXiv:https://doi.org/10.1080/09687599.2020.1836478 doi:10. 1080/09687599.2020.1836478 - source
- cites [88] [88] Erin Pritchard. 2024. Let's Tackle Representations of Dwarfism in the Media . https://www.hope.ac.uk/news/allnews/lets-tackle-representations-ofdwarfism-in-the-media.html Liverpool Hope University News. - source
- cites [114] [114] Susan Vertoont, Tina Goethals, Frederik Dhaenens, Patrick Schelfhout, Tess Van Deynse, Gabria Vermeir, and Maud Ysebaert. 2022. Un/recognisable and dis/empowering images of disability: A collective textual analysis of media representations of intellectual disabilities. Critical Studies in Media Communication 39, 1 (2022), 1-14. - source
- cites [117] [117] Alison Wilde. 2022. The representation of disabled women and recent disabled women-led media. Disability & Society 37, 3 (2022), 522-527. - source
The Introduction says disabled women were noticeably absent from screens and other media less than three decades ago, and describes a television programme that gave them visibility while promoting dubious ideas about them. It also discusses the limited influence disabled people have over media stories. These passages support concerns about absence and portrayals, but focus chiefly on disabled women rather than establishing the full claim about all people with disabilities or characterizing portrayals generally as harmful or inaccurate. (LLM, conf=0.85)
Experimental — accuracy not yet measured
- Premisecitation§ 1 Introduction · p. 2
Approaches that enable communities to define and curate datasets remain underexplored.
“Despite calls for more inclusive data-centric AI (e.g., [11, 31, 53, 96, 107]), approaches that enable communities to define and curate datasets remain underexplored.”
Quote in the paperSubstantial claim3 of 5 sources checked- cites [11] [11] Stevie Bergman, Nahema Marchal, John Mellor, Shakir Mohamed, Iason Gabriel, and William Isaac. 2024. STELA: a community-centred approach to norm elicitation for AI alignment. Scientific Reports 14, 1 (2024), 6616. - source
The Introduction calls for greater inclusion of historically marginalised communities in AI development, and the Methods say participatory approaches to AI alignment are still in their early stages. This supports the cited sentence’s general point about inclusive AI, but the paper discusses community-defined rules for AI behaviour, not communities defining or curating datasets. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [31] [31] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23) . Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages. doi:10.1145/3617694.3623261 - source
The Introduction describes growing calls for stakeholder participation in AI design, including data generation and annotation. The Discussion (§4) finds that only a handful of surveyed projects let stakeholders drive or own any part of design. This supports the broader concern about limited community agency, but the paper does not specifically establish that approaches for communities to define and curate datasets remain underexplored. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
- cites [96] [96] Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. 'Everyone wants to do the model work, not the data work': Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-15. - source
- cites [107] [107] Mei Tan, Hansol Lee, Dakuo Wang, and Hari Subramonyam. 2024. Is a seat at the table enough? Engaging teachers and students in dataset specification for ml in education. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024), 1-32. -
Not checked in the free run.
- Rests on an assumption: Community involvement in AI data practices, if present, has often been limited to data collection, annotation or model output evaluation.
- Rests on an assumption: Conceptualized representation and its translation in data are under-supported.
- Premisecitation§ 1 Introduction · p. 2
AI image generation models currently depict people with disabilities in unrealistic or dehumanizing ways.
“As a consequence, they currently depict PwD in unrealistic or dehumanizing ways [48, 77], and often omit or distort representations of assistive technologies [77].”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [48] [48] Kate S Glazko, Momona Yamagami, Aashaka Desai, Kelly Avery Mack, Venkatesh Potluri, Xuhai Xu, and Jennifer Mankoff. 2023. An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY, USA) (ASSETS '23) . Association for Computing Machinery, New York, NY, USA, Article 99, 8 pages. doi:10.1145/3597638.3614548 - source
Vignette 3 reports that Midjourney generated images that did not represent many disabilities accurately, and the Practical Guidance section describes difficulty producing a representative image of disabled people. This supports the claim about unrealistic depictions, but the paper’s image-generation evidence comes from a Midjourney case study, not image-generation models generally; it does not characterize the images as dehumanizing. (LLM, conf=0.83)
Experimental — accuracy not yet measured - cites [77] [77] Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K. Kane, and Cynthia L. Bennett. 2024. 'They only care to show us the wheelchair': disability representation in text-to-image AI models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 288, 23 pages. doi:10.1145/3613904.3642166 - source
- Premisecitation§ 1 Introduction · p. 2
Media presentation of people with disabilities influences their livelihood through access to education and employment.
“How people with disabilities (PwD) are presented in media plays a pivotal role in shaping public perceptions and social norms [34, 105], influencing livelihood through access to education and employment [47].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [47] [47] Kate Glazko, Yusuf Mohammed, Ben Kosa, Venkatesh Potluri, and Jennifer Mankoff. 2024. Identifying and Improving Disability Bias in GPT-Based Resume Screening. In The 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT '24) . ACM, 687-700. doi:10.1145/3630106.3658933 - source
Section 7.1 (p. 14) describes discrimination against disabled jobseekers and disparities in employment, salaries, and career advancement, supporting the employment-related part of the cited clause. The paper does not establish that media portrayals affect access to education or employment; its discussion of stereotypes in Section 4.4 (p. 12) concerns GPT-4 resume screening, not media presentation. (LLM, conf=0.83)
Experimental — accuracy not yet measured
- Premisecitation§ 1 Introduction · p. 2
AI media generation models have been trained on datasets that reflect historical erasure and bias.
“AI media generation, specifically image generation models that translate free-form text prompts into visuals [87, 94, 95], have been trained on datasets that reflect this historical erasure and bias.”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [87] [87] Nikita Pavlichenko and Dmitry Ustalov. 2023. Best prompts for text-to-image models and how to find them. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval . 2067-2071. - source
- cites [94] [94] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In International conference on machine learning . Pmlr, 8821-8831. -
Not checked in the free run.
- cites [95] [95] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems 35 (2022), 36479-36494. - source
- Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
Common image generation evaluation metrics such as CLIPScore, FID, and uses of MLLMs 'as-judge' metrics often penalize authentic depictions of assistive technologies, while rewarding incorrect or even offensive renderings.
“However, common image generation evaluation metrics -such as CLIPScore [55], FID [56], and uses of multimodal large language models (MLLMs) 'as-judge' metrics [58, 71] often penalize authentic depictions of ATs, while rewarding incorrect or even offensive renderings (e.g., Braille displays shown as paper) [65].”
Quote in the paperSubstantial claim3 of 4 sources checked- cites [55] [55] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718 (2021). - source
The Introduction and “Evaluating Caption Generations with CLIP” identify CLIPScore as an evaluation metric, supporting the citation’s identification of the metric. The paper evaluates generated captions for images, not generated images, so it does not establish the cited description of CLIPScore as an image-generation evaluation metric. The claim about assistive-technology depictions is attached to [65], not this citation. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [56] [56] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017). - source
- cites [58] [58] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. 2023. Tifa: Accurate and interpretable text-toimage faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 20406-20417. -
Not checked in the free run.
- cites [71] [71] Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. 2023. Viescore: Towards explainable metrics for conditional image synthesis evaluation. arXiv preprint arXiv:2312.14867 (2023). - source
Sections 3.3 and 5.3 describe limitations of CLIPScore, including low scores for images human raters judged to have high semantic consistency; Section 5.2 describes MLLMs missing image details. The paper does not establish that these metrics penalize authentic assistive-technology depictions or reward incorrect or offensive renderings, including the Braille-display example. (LLM, conf=0.82)
Experimental — accuracy not yet measured
- Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
Web-scraped data for AI model pre-training is skewed towards an underrepresentation of disability data.
“Web-scraped data for AI model pre-training is skewed towards an underrepresentation of disability data [69, 79].”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [69] [69] Elisa Kreiss, Cynthia Bennett, Shayan Hooshmand, Eric Zelikman, Meredith Ringel Morris, and Christopher Potts. 2022. Context matters for image descriptions for accessibility: Challenges for referenceless evaluation metrics. arXiv preprint arXiv:2205.10646 (2022). - source
Section 2.1, “Image Accessibility” (page not provided), reports that image descriptions are rare on some websites, but does not discuss the representation of disability data in AI pre-training datasets. Searches for web-scraped pre-training data, disability underrepresentation, and training-data bias found no passage addressing the claim. (LLM, conf=0.75)
Experimental — accuracy not yet measured - cites [79] [79] Daniela Massiceti, Camilla Longden, Agnieszka Słowik, Samuel Wills, Martin Grayson, and Cecily Morrison. 2024. Explaining CLIP's performance disparities on data from blind/low vision users. arXiv:2311.17315 [cs.CV] https://arxiv.org/ abs/2311.17315 - source
- Premisecitation§ 1 Introduction · p. 2
Unlike other forms of marginalization (e.g., gender, race), disability representation has yet to be publicly debated or defined.
“Unlike other forms of marginalization (e.g., gender, race), disability representation has yet to be publicly debated or defined (cf. evolving work by [103, 117]).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [103] [103] Jonatan Södergren and Niklas Vallström. 2023. Disability in influencer marketing: a complex model of disability representation. Journal of Marketing Management 39, 11-12 (2023), 1012-1042. - source
The Introduction says disability is difficult to define and calls for more nuance in its representation. It does not say disability representation has yet to be publicly debated, or compare that debate with gender or race. It also reviews existing models of disability representation, so the claim that representation has not been defined overstates the source. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [117] [117] Alison Wilde. 2022. The representation of disabled women and recent disabled women-led media. Disability & Society 37, 3 (2022), 522-527. - source
The Abstract says portrayals of disabled women have been disproportionately neglected in academic analysis, and the Introduction discusses their changing screen presence. Neither section establishes that disability representation has yet to be publicly debated or defined, or compares that debate with gender and race; searches for those points found no supporting passage in our copy. (LLM, conf=0.72)
Experimental — accuracy not yet measured
- Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3
Visual media plays a significant role in shaping public opinion and social norms, including societal perceptions of disability.
“Visual media plays a significant role in shaping public opinion and social norms, including societal perceptions of disability [10, 34, 105, 117].”
Quote in the paperSubstantial claim1 of 4 sources checked- cites [10] [10] Cynthia L Bennett, Shaun K Kane, and Christina N Harrington. 2025. Toward Community-Led Evaluations of Text-to-Image AI Representations of Disability, Health, and Accessibility. In Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization . 256-270. - source
- cites [34] [34] Katie Ellis, Gerard Goggin, Beth A. Haller, and Rosemary Curtis (Eds.). 2020. The Routledge Companion to Disability and Media . Routledge, New York. https: //doi.org/10.4324/9781315716008 - source
- cites [105] [105] David Welch Suggs and Jason Lee Guthrie. 2017. Disabling Prejudice: A Case Study of Images of Paralympic Athletes and Attitudes Toward People with Disabilities. International Journal of Sport Communication 10, 2 (2017), 258-276. https://www.researchgate.net/publication/317286559\_Disabling\_Prejudice\_ A\_Case\_Study\_of\_Images\_of\_Paralympic\_Athletes\_and\_Attitudes\_Toward\_ People\_With\_Disabilities - source
- cites [117] [117] Alison Wilde. 2022. The representation of disabled women and recent disabled women-led media. Disability & Society 37, 3 (2022), 522-527. - source
- Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
Smaller datasets can be vital in enabling key post-training techniques such as fine-tuning, in-context learning, and prompt engineering.
“Nevertheless, smaller datasets can be vital in enabling key post -training techniques such as fine-tuning, in-context learning, and prompt engineering (cf. [86, 122]).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [86] [86] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730-27744. - source
- cites [122] [122] Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. Advances in Neural Information Processing Systems 36 (2023), 5500655021. source
The Introduction and Discussion report that fine-tuning a pretrained language model on 1,000 carefully curated examples produced strong results. The cited paper does not establish that small datasets enable in-context learning or prompt engineering, the other techniques named in the sentence. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
Even when people with disabilities data is available, data from marginalized groups is often automatically filtered out by techniques for removing 'low' quality data, which rely on biased models such as CLIP.
“Even when PwD data is available, data from marginalized groups is often automatically filtered out by techniques for removing 'low' quality data [57, 104], which rely on biased models such as CLIP [92].”
Quote in the paperSubstantial claim1 of 3 sources checked- cites [57] [57] Rachel Hong, William Agnew, Tadayoshi Kohno, and Jamie Morgenstern. 2024. Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp. In Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization . 1-17. - source
- cites [104] [104] Hariharan Subramonyam, Colleen Seifert, and MI Eytan Adar. 2021. How can human-centered design shape data-centric AI. In Proceedings of NeurIPS Data-Centric AI Workshop . Curran Associates New York, NY, USA, 3. -
Not checked in the free run.
- cites [92] [92] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning . PmLR, 8748-8763. -
Not checked in the free run.
- Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
Previous research has highlighted challenges for image generators to depict people with disabilities in respectful and realistic ways.
“Previous research has highlighted challenges for image generators to depict PwD in respectful and realistic ways [48, 77], echoing broader biases seen in representations of other marginalized communities [12, 21, 81, 91].”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [48] [48] Kate S Glazko, Momona Yamagami, Aashaka Desai, Kelly Avery Mack, Venkatesh Potluri, Xuhai Xu, and Jennifer Mankoff. 2023. An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY, USA) (ASSETS '23) . Association for Computing Machinery, New York, NY, USA, Article 99, 8 pages. doi:10.1145/3597638.3614548 - source
- cites [77] [77] Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K. Kane, and Cynthia L. Bennett. 2024. 'They only care to show us the wheelchair': disability representation in text-to-image AI models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 288, 23 pages. doi:10.1145/3613904.3642166 - source
- Premisecitation§ 1 Introduction · p. 2
Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups, including people with disabilities.
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
Quote in the paperSubstantial claim8 of 10 sources checked- cites [15] [15] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems , D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc. https://proceedings.neurips.cc/paper\_files/ paper/2016/file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf - source
The discussion on p. 3 says word embeddings reflect societal gender stereotypes and can amplify them; it also gives stereotyped analogy outputs. This supports the bias point, but the paper studies word embeddings, not generative AI outputs generally or biases affecting marginalized groups broadly. The separate PwD clause has its own citations and is not attributed to this source. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [21] [21] Marc Cheong, Ehsan Abedin, Marinus Ferreira, Ritsaart Reimann, Shalom Chalson, Pamela Robinson, Joanne Byrne, Leah Ruppanner, Mark Alfano, and Colin Klein. 2024. Investigating Gender and Racial Biases in DALL-E Mini Images. ACM J. Responsib. Comput. 1, 2, Article 13 (June 2024), 20 pages. - source
- cites [36] [36] Anjalie Field, Amanda Coston, Nupoor Gandhi, Alexandra Chouldechova, Emily Putnam-Hornstein, David Steier, and Yulia Tsvetkov. 2023. Examining risks of racial biases in NLP tools for child protective services. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 1479-1492. doi:10.1145/3593013.3594094 - source
The Abstract says NLP models can absorb and amplify data biases, including biases related to race and disability. But the Discussion does not establish that generative AI outputs tend to reproduce normative views or societal biases; it calls for evaluating generative models for racial bias if they are deployed. The source therefore supports the broader bias concern, not the claim specifically about generative outputs. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [41] [41] Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences 115, 16 (April 2018). doi:10.1073/pnas. 1720347115 - source
The Discussion (p. 8) finds that word embeddings reflect gender and ethnic biases and associations about how groups are viewed. It does not examine generative AI outputs or establish that those outputs reproduce such biases. The sentence cites a separate group of sources for its claim about people with disabilities. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [42] [42] Sanjana Gautam, Pranav Narayanan Venkit, and Sourojit Ghosh. 2024. From melting pots to misrepresentations: Exploring harms in generative ai. arXiv preprint arXiv:2403.10776 (2024). - source
- cites [45] [45] Tarleton Gillespie. 2024. Generative AI and the politics of visibility. Big Data & Society 11, 2 (2024), 20539517241252131. - source
- cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
- cites [66] [66] Samia Kabir, Lixiang Li, and Tianyi Zhang. 2024. STILE: Exploring and Debugging Social Biases in Pre-trained Text Representations. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 293, 20 pages. doi:10.1145/3613904.3642111 - source
The Introduction (page not available) says language models trained on real-world text risk propagating or amplifying social biases and that such models are used for text generation. This supports the general bias concern, but the paper frames propagation as a risk rather than establishing that generative AI outputs tend to reproduce normative views. The sentence cites other sources separately for the claim about people with disabilities. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [72] [72] Xingyu Lan, Jiaxi An, Yisu Guo, Tong Chiyou, Xintong Cai, and Jun Zhang. 2025. Imagining the Far East: Exploring Perceived Biases in AI-Generated Images of East Asian Women. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25) . Association for Computing Machinery, New York, NY, USA, Article 332, 7 pages. doi:10. 1145/3706599.3719678 - source
- cites [91] [91] Rida Qadri, Renee Shelby, Cynthia L. Bennett, and Remi Denton. 2023. AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 506-517. doi:10.1145/3593013. 3594016 - source
- Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 3
There is a growing trend in proxy-based participation, whereby individuals familiar with a stakeholder community, including UX/HCI practitioners, often 'stand-in' for others rather than enabling communities to speak for themselves.
“There is also a growing trend in proxy-based participation, whereby individuals familiar with a stakeholder community, including UX/HCI practitioners, often 'stand-in' for others [27, 31, 106] rather than enabling communities to speak for themselves.”
Quote in the paperSubstantial claim2 of 3 sources checked- cites [27] [27] Ned Cooper and Alexandra Zafiroglu. 2024. From Fitting Participation to Forging Relationships: The Art of Participatory ML. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 746, 9 pages. doi:10.1145/3613904.3642775 - source
Section 4.1 (p. 4) describes brokers who sit between ML systems and affected people, and reports difficulty engaging indirect stakeholders. But the paper does not establish a growing trend of UX/HCI practitioners or other community-familiar individuals standing in for communities rather than enabling them to speak for themselves. Its discussion of proxies in Section 2.3 (p. 2) concerns algorithmic proxies, not those individuals. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [31] [31] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23) . Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages. doi:10.1145/3617694.3623261 - source
- cites [106] [106] Cella M Sum, Franchesca Spektor, Rahaf Alharbi, Leya Breanna Baltaxe-Admony, Erika Devine, Hazel Anneke Dixon, Jared Duval, Tessa Eagle, Frank Elavsky, Kim Fernandes, et al. 2024. Challenging ableism: A critical turn toward disability justice in hci. XRDS: Crossroads, The ACM Magazine for Students 30, 4 (2024), 50-55. - source
- Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 3
Delgado et al. found that most AI projects bring stakeholders merely in as consultants, in one-off preference elicitation or UI decisions.
“However, in a recent review paper, Delgado et al. [31] found that most AI projects bring stakeholders merely in as consultants, in one-off preference elicitation or UI decisions (cf. also [27, 108]).”
Quote in the paperSubstantial claim3 of 3 sources checked- cites [31] [31] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23) . Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages. doi:10.1145/3617694.3623261 - source
- cites [27] [27] Ned Cooper and Alexandra Zafiroglu. 2024. From Fitting Participation to Forging Relationships: The Art of Participatory ML. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 746, 9 pages. doi:10.1145/3613904.3642775 - source
Section 2.3 (p. 2) reports a tendency in participatory ML projects to inform or consult participants rather than form equitable partnerships, citing Delgado et al. for that point. But this source is Cooper et al.’s interview study, not a review by Delgado et al.; it does not establish that most AI projects use one-off consultations or limit stakeholders to UI decisions. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [108] [108] Mei Tan, Hansol Lee, Dakuo Wang, and Hari Subramonyam. 2024. Is a Seat at the Table Enough? Engaging Teachers and Students in Dataset Specification for ML in Education. Proc. ACM Hum.-Comput. Interact. 8, CSCW1, Article 81 (April 2024), 32 pages. doi:10.1145/3637358 - source
Section 2.3 (page unavailable) says that, in educational AI, teachers and students are often consulted through need-finding interviews or product-design feedback and engaged only in a limited capacity as end users. It does not establish that most AI projects use one-off preference elicitation or UI decisions, and the cited paper is Tan et al.’s study, not a review by Delgado et al. (LLM, conf=0.85)
Experimental — accuracy not yet measured
- Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4
The identity and views of human annotators become embedded in the data, influencing subsequent system behavior.
“In expressing 'their' perspective through data annotations, it important to recognize how the identity and views of human annotators become embedded in the data, influencing subsequent system behavior [51, 52, 90].”
Quote in the paperSubstantial claim3 of 3 sources checked- cites [51] [51] Nitesh Goyal, Ian D Kivlichan, Rachel Rosen, and Lucy Vasserman. 2022. Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1-28. - source
- cites [52] [52] Melissa Hall, Samuel J Bell, Candace Ross, Adina Williams, Michal Drozdzal, and Adriana Romero Soriano. 2024. Towards geographic inclusion in the evaluation of text-to-image models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 585-601. - source
The Introduction (p. 3) and Geographic Representation results (pp. 7–9) show that annotators’ locations and perspectives shape their judgments, supporting the point that those views are reflected in annotations. The paper compares annotations with evaluations of existing text-to-image models; it does not show that the annotations influence subsequent system behavior. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [90] [90] Rida Qadri, Michael Madaio, and Mary L. Gray. 2025. Confusing the Map for the Territory. Commun. ACM (Sept. 2025), 4 pages. doi:10.1145/3731655 Online First. - source
- Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4
The most dominant approach to gathering human annotations is to 'crowd-source' either anonymous people or community members as 'data workers', who are given simple, tokenistic tasks with little or no context to the data.
“The most dominant approach to gathering human annotations is to 'crowd-source' [7, 53, 65] either anonymous people or community members as 'data workers', who are given simple, tokenistic tasks 'that even inexperienced annotators can finish in an instance' [31, 85], with little or no context to the data.”
Quote in the paperSubstantial claim4 of 5 sources checked- cites [7] [7] Abhipsa Basu, R Venkatesh Babu, and Danish Pruthi. 2023. Inspecting the geographical representativeness of images from text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 5136-5147. - source
- cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
- cites [65] [65] Nari Johnson, Hamna ., Deepthi Sudharsan, Theo Holroyd, Samantha Dalal, Siobhan Mackenzie Hall, Jennifer Wortman Vaughan, Daniela Massiceti, and Cecily Morrison. 2025. Position: To Make Text-to-Image Models that Work for Marginalized Communities, We Need New Measurement Practices for the Long Tail. (July 2025). https://www.microsoft.com/en-us/research/wpcontent/uploads/2025/07/longtail.pdf. -
Not checked in the free run.
- cites [31] [31] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23) . Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages. doi:10.1145/3617694.3623261 - source
The Introduction describes many practitioners involving stakeholders as “data labor,” including in data labeling or annotation. It does not establish that crowdsourcing is the most dominant way to gather annotations, or describe annotators as anonymous people or community members given little context. The quotation about simple tasks is attached to different citations in the citing sentence. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [85] [85] Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkilä, and Shin'ichi Satoh. 2023. Toward verifiable and reproducible human evaluation for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 14277-14286. - source
- Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 3
In research involving people with disabilities, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems.
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of accesssupporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
Quote in the paperSubstantial claim3 of 8 sources checked- cites [2] [2] Rudaiba Adnin and Maitraye Das. 2024. "I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John's, NL, Canada) (ASSETS '24) . Association for Computing Machinery, New York, NY, USA, Article 64, 14 pages. doi:10.1145/3663548.3675631 - source
- cites [3] [3] Rahaf Alharbi, Pa Lor, Jaylin Herskovitz, Sarita Schoenebeck, and Robin N. Brewer. 2024. Misfitting With AI: How Blind People Verify and Contest AI Errors. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John's, NL, Canada) (ASSETS '24) . Association for Computing Machinery, New York, NY, USA, Article 61, 17 pages. doi:10. 1145/3663548.3675659 - source
The Method and Introduction describe interviews with 26 blind users about errors in access-supporting AI visual assistance technologies and features they would like improved. That supports participant input on existing systems, but the paper does not describe participants collaborating on system design or a formal evaluation, nor establish that such collaboration characterizes most disability research. The technologies studied are also not described as exclusively generative AI. (LLM, conf=0.83)
Experimental — accuracy not yet measured - cites [22] [22] Arnavi Chheda-Kothary, Ritesh Kanchi, Chris Sanders, Kevin Xiao, Aditya Sengupta, Melanie Kneitmix, Jacob O. Wobbrock, and Jon E. Froehlich. 2025. ArtInsight: Enabling AI-Powered Artwork Engagement for Mixed Visual-Ability Families. In Proceedings of the 30th International Conference on Intelligent User Interfaces (IUI '25) . Association for Computing Machinery, New York, NY, USA, 190-210. doi:10.1145/3708359.3712082 - source
- cites [60] [60] Shuxu Huffman, Si Chen, Kelly Avery Mack, Haotian Su, Qi Wang, and Raja Kushalnagar. 2025. "We do use it, but not how hearing people think": How the Deaf and Hard of Hearing Community Uses Large Language Model Tools. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25) . Association for Computing Machinery, New York, NY, USA, Article 33, 9 pages. doi:10.1145/3706599.3719785 - source
The Abstract describes a study in which Deaf and Hard of Hearing participants reported how they use Text AI, and §3.2 says interviewees tried ChatGPT tasks and assessed its responses. The paper supports participant input on the accessibility of an existing generative AI tool, but it does not describe participants collaborating to design an access-supporting system or establish that this is what participants mostly do across disability research. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [61] [61] Mina Huh, Yi-Hao Peng, and Amy Pavel. 2023. GenAssist: Making Image Generation Accessible. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (San Francisco, CA, USA) (UIST '23) . Association for Computing Machinery, New York, NY, USA, Article 38, 17 pages. doi:10.1145/3586183.3606735 - source
- cites [73] [73] Seonghee Lee, Maho Kohga, Steve Landau, Sile O'Modhrain, and Hari Subramonyam. 2024. AltCanvas: A Tile-Based Editor for Visual Content Creation with Generative AI for Blind or Visually Impaired People. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John's, NL, Canada) (ASSETS '24) . Association for Computing Machinery, New York, NY, USA, Article 70, 22 pages. doi:10.1145/3663548.3675600 - source
- cites [98] [98] JooYoung Seo, Sanchita S. Kamath, Aziz Zeidieh, Saairam Venkatesh, and Sean McCurry. 2024. MAIDR Meets AI: Exploring Multimodal LLM-Based Data Visualization Interpretation by and with Blind and Low-Vision Users. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John's, NL, Canada) (ASSETS '24) . Association for Computing Machinery, New York, NY, USA, Article 57, 31 pages. doi:10.1145/3663548.3675660 - source
- cites [109] [109] Xinru Tang, Ali Abdolrahmani, Darren Gergle, and Anne Marie Piper. 2025. Everyday Uncertainty: How Blind People Use GenAI Tools for Information Access. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25) . Association for Computing Machinery, New York, NY, USA, Article 63, 17 pages. doi:10.1145/3706598.3713433 - source
- Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4
Hall et al. found that annotators who were not from the geographical region depicted in AI images tended to favor exaggerated, stereotypical images and overlooked other realistic, representative portrayals.
“For example, Hall et al. [52] found that annotators, who were not from the geographical region depicted in AI images tended to favor exaggerated, stereotypical images and overlooked other realistic, representative portrayals.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [52] [52] Melissa Hall, Samuel J Bell, Candace Ross, Adina Williams, Michal Drozdzal, and Adriana Romero Soriano. 2024. Towards geographic inclusion in the evaluation of text-to-image models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 585-601. - source
- Findingcitation§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15
Each community actively negotiated their own representation goals, shaped by their social, geographical and cultural contexts, and in conversation with community members and image materials.
“Echoing recent research by Qadri et al. [89], our findings show how each community actively negotiated their own representation goals -shaped by their social, geographical and cultural contexts, and in conversation with community members and image materials.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [89] [89] Rida Qadri, Mark Diaz, Ding Wang, and Michael Madaio. 2025. The case for" thick evaluations" of cultural representation in ai. arXiv preprint arXiv:2503.19075 (2025). - source
- Interpretationcitation§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15
Social worlds and their ideal representations are not static, but can evolve over time in response to changing contexts and aspirations.
“Furthermore, our findings surface how social worlds and their ideal representations are not static, but can evolve over time in response to changing contexts and aspirations (cf. [89]).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [89] [89] Rida Qadri, Mark Diaz, Ding Wang, and Michael Madaio. 2025. The case for" thick evaluations" of cultural representation in ai. arXiv preprint arXiv:2503.19075 (2025). - source
- Interpretationcitation§ 5.3 Human-Centred AI Data Work · p. 17
Rather than treating PwD as a singular or universal identity, the authors acknowledge the distinct social identities and intersectionalities within each community.
“Rather than treating PwD as a singular or universal identity -often reinforced by homogenizing labels such as 'dwarfism -we acknowledge the distinct social identities and intersectionalities within each community (cf. [53, 90]).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
Section 6.3 says distinct cultures should not be lumped into a “single flattened snapshot” and describes an underrepresented ethnic community within Nigeria. That supports the broader point about avoiding homogenizing labels, but this paper addresses cultural representation, not PwD identities or their intersectionalities. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [90] [90] Rida Qadri, Michael Madaio, and Mary L. Gray. 2025. Confusing the Map for the Territory. Commun. ACM (Sept. 2025), 4 pages. doi:10.1145/3731655 Online First. - source
- Findingown data§ 4.1 Defining Representation · p. 10
Almost all images P1 selected for education depicted important ATs for his community 'in-use' and across various learning contexts.
“Consequently, almost all images P1 selected for education depicted important ATs for his community 'in-use' and across various learning contexts -such as a range of curricular activities, and different learning dynamics (e.g., a blind teacher passing their skills on to a VI student; a VI learner with sighted peers in a classroom).”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 11
Community engagements included portraying PwD in professional roles to foster positive narratives.
“Across the organizations, these engagements re-emphasized previous and surfaced new themes, including: portraying PwD in professional roles to foster positive narratives (P1, P2, P3); challenging misconceptions of PwD not having the right, or being able to raise a family (P1, P2); and ensuring accurate depictions of bodies or devices (P1, P2).”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 10
Project leads navigated choices about meaningful variations by deliberating what to capture and what aspects to prioritize (for AI) in discussions with other community members.
“Project leads navigated making those choices by deliberating what meaningful variations to capture and what aspects to prioritize (for AI) in discussions with other community members .”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 10
There are geographical and cultural differences in representational priorities.
“Interlinked with this, we observed geographical and cultural differences in representational priorities.”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Defining Representation · p. 9
All project leads tended to articulate their representation aspirations through image instances that would implicitly or explicitly challenge existing misconceptions and common stereotypes, aiming to counter negative or limiting narratives of their community in defining positive aspirations.
“Firstly, we saw that all project leads tended to articulate their representation aspirations through image instances that would implicitly or explicitly challenge existing misconceptions and common stereotypes -aiming to counter 'negative' or 'limiting' narratives of their community in defining positive aspirations .”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 11
Community engagements included challenging misconceptions of PwD not having the right, or being able to raise a family.
“Across the organizations, these engagements re-emphasized previous and surfaced new themes, including: portraying PwD in professional roles to foster positive narratives (P1, P2, P3); challenging misconceptions of PwD not having the right, or being able to raise a family (P1, P2); and ensuring accurate depictions of bodies or devices (P1, P2).”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 10
A practical challenge in envisioning a more inclusive future lies in the scarcity of authentic imagery, particularly for activities that are yet to be realized or rarely documented.
“A very practical challenge in envisioning a more inclusive future also lies in the scarcity of authentic imagery, particularly for activities that are yet to be realized, or rarely documented.”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Defining Representation · p. 11
Across the organizations, community engagements re-emphasized previous and surfaced new themes.
“Across the organizations, these engagements re-emphasized previous and surfaced new themes, including: portraying PwD in professional roles to foster positive narratives (P1, P2, P3); challenging misconceptions of PwD not having the right, or being able to raise a family (P1, P2); and ensuring accurate depictions of bodies or devices (P1, P2).”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 11
For dwarfism, community members emphasized showing the right body proportions.
“For dwarfism, this meant showing the right body proportions (P2); for VI, community members emphasized realistic eye representation -avoiding distorted or enlarged depictions -and the correct depiction and handling of ATs (P1).”
Quote in the paperNot a substantial claim - Findingreasoning§ 4.1 Defining Representation · p. 10
The Pinboard activity encouraged project leads to imagine aspirational representations that expand what is typically seen or considered possible today.
“This highlights how familiar visual norms can constrain representation , and demonstrates how the Pinboard activity encouraged project leads to imagine aspirational representations that expand what is typically seen or considered possible today .”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 10
Project leads began to actively define the boundaries of their communities.
“Lastly, we observed how project leads began to actively define the boundaries of their communities.”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 11
Providing educational materials equipped project leads to more confidently explain core concepts and motivate participation.
“These resources equipped project leads to more confidently explain core concepts and motivate participation.”
Quote in the paperSubstantial claim - Findingown data§ 3.1 Disability Communities & Project Leads · p. 4
A manual, human review by two disability experts of images generated for 662 prompts (612 benign, 50 adversarial), prompted across ten main types of disabilities using an off-the-shelf image generation service found that defect rates were higher for Dwarfism and Blind and Low Vision communities.
“A manual, human review by two disability experts of images generated for 662 prompts (612 benign, 50 adversarial), prompted across ten main types of disabilities (e.g. mental health, blindness, low vision, mobility, neurodiversity, learning, speech, deafness, hard of hearing, pan disability) using an off-the-shelf image generation service found that defect rates were higher for Dwarfism and Blind”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Defining Representation · p. 10
The organizations varied most in how they conceptualized 'diversity'.
“The organizations varied most in how they conceptualized 'diversity'.”
Quote in the paperNot a substantial claim - Findingown data§ 4.1 Defining Representation · p. 11
Community feedback was central to shaping representation definitions.
“Community feedback was central to shaping representation definitions.”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Defining Representation · p. 11
The Magazine prompted project leads to iterate or reframe what to include in images.
“Finally, the Magazine prompted project leads to iterate or reframe what to include in images.”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Defining Representation · p. 11
Community engagements included ensuring accurate depictions of bodies or devices.
“Across the organizations, these engagements re-emphasized previous and surfaced new themes, including: portraying PwD in professional roles to foster positive narratives (P1, P2, P3); challenging misconceptions of PwD not having the right, or being able to raise a family (P1, P2); and ensuring accurate depictions of bodies or devices (P1, P2).”
Quote in the paperNot a substantial claim - Findingown data§ 3.1 Disability Communities & Project Leads · p. 4
The authors focused on disability organizations representing people with dwarfism and people with vision impairments as two disability groups for whom internal evaluations showed image generation model outputs to be particularly poor.
“We focused on disability organizations that specifically represent people with dwarfism and people with vision impairments as two disability groups for whom internal evaluations 1 showed image generation model outputs to be particularly poor.”
Quote in the paperSubstantial claim - Findingown data§ 4.1 Defining Representation · p. 10
Project leads faced challenges in balancing representations of the community's current, lived experiences and imagined future aspirations.
“Secondly, we observed challenges in balancing representations of the community's current, lived experiences and imagined future aspirations.”
Quote in the paperNot a substantial claim - Findingown data§ 4.2 Curating the Community Library · p. 12
Project leads appreciated the reflective 'why' question as a clear, non-technical way to describe image relevance.
“For image annotations, project leads appreciated the reflective 'why' question as a clear, non-technical way to describe image relevance.”
Quote in the paperNot a substantial claim - Findingown data§ 4.3 Evaluating AI · p. 14
Images were rated as 'bad' or 'very bad' where the disability was either not recognizable or mis-portrayed.
“Images were rated as 'bad' or 'very bad', where the disability was either not recognizable or mis-portrayed .”
Quote in the paperNot a substantial claim - Findingnone given§ 4.2 Curating the Community Library · p. 12
Organizations navigated practical constraints in capturing desired, diverse representations-especially securing AI-use 'consent' for images of 'groups' or 'children'.
“Finally, organizations navigated practical constraints in capturing desired, diverse representations -especially securing AI-use 'consent' for images of 'groups' or 'children'.”
Quote in the paperNot a substantial claim - Findingown data§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 15
Reflection scaffolds and data structures provided through the Community Library Creator served as a productive bridge for project leads to articulate their communities' representation goals in data.
“Our findings show how reflection scaffolds and data structures provided through the Community Library Creator -such as Pinboard, Magazine, the Community Library, image annotations, and AI image ratings -served as a productive bridge for project leads to articulate their communities' representation goals in data.”
Quote in the paperSubstantial claim - Findingown data§ 5.3 Human-Centred AI Data Work · p. 17
The authors saw how organizations deliberated issues of consent and negotiated the boundaries of their community with their members.
“We saw how they deliberated issues of consent and negotiated the boundaries of their community with their members.”
Quote in the paperSubstantial claim - Findingown data§ 5.3 Human-Centred AI Data Work · p. 17
In their research, differences in representation priorities, disability language, and scope of diversity of the two dwarfism communities were most evident.
“In our research, this was most evident in differences in the representation priorities, disability language, and scope of diversity of the two dwarfism communities.”
Quote in the paperSubstantial claim - Findingown data§ 5.3 Human-Centred AI Data Work · p. 17
Engaging with disability advocacy organizations as proxies for bounded communities provides a practical and realistic approach to involving non-AI experts in AI data practices that respects each community's own ways of organizing themselves and their culturally embedded power dynamics.
“We found that engaging with disability advocacy organizations as proxies for bounded communities provides a practical and realistic approach to involving non-AI experts in AI data practices that respects each communities' own ways of organizing themselves, and their culturally embedded power dynamics.”
Quote in the paperSubstantial claim - Findingnone given§ 5.3 Human-Centred AI Data Work · p. 17
Advocacy organizations already have established processes to support communication and decision-making, enabling them to address tensions and differing perspectives across their membership.
“Advocacy organizations also already have established processes to support communication and decision-making, enabling them to address tensions and differing perspectives across their membership.”
Quote in the paperSubstantial claim - Premisecitation§ 1 Introduction · p. 2
How people with disabilities are presented in media plays a pivotal role in shaping public perceptions and social norms.
“How people with disabilities (PwD) are presented in media plays a pivotal role in shaping public perceptions and social norms [34, 105], influencing livelihood through access to education and employment [47].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [34] [34] Katie Ellis, Gerard Goggin, Beth A. Haller, and Rosemary Curtis (Eds.). 2020. The Routledge Companion to Disability and Media . Routledge, New York. https: //doi.org/10.4324/9781315716008 - source
- cites [105] [105] David Welch Suggs and Jason Lee Guthrie. 2017. Disabling Prejudice: A Case Study of Images of Paralympic Athletes and Attitudes Toward People with Disabilities. International Journal of Sport Communication 10, 2 (2017), 258-276. https://www.researchgate.net/publication/317286559\_Disabling\_Prejudice\_ A\_Case\_Study\_of\_Images\_of\_Paralympic\_Athletes\_and\_Attitudes\_Toward\_ People\_With\_Disabilities - source
- Premisecitation§ 1 Introduction · p. 2
AI-generated visual media is becoming rapidly more accessible to people.
“As AI-generated visual media becomes rapidly more accessible to people [81], there is opportunity to address this historical misrepresentation and promote a more authentic, positive representation of disability.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [81] [81] Nusrat Jahan Mim, Dipannita Nandi, Sadaf Sumyia Khan, Arundhuti Dey, and Syed Ishtiaque Ahmed. 2024. In-between visuals and visible: The impacts of text-to-image generative ai tools on digital image-making practices in the global south. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems . 1-18. - source
- Premisecitation§ 1 Introduction · p. 2
One billion people with disabilities globally remain largely invisible or inaccurately depicted in the media.
“Although visibility of PwD is slowly increasing, one billion PwD globally remain largely invisible or inaccurately depicted in the media [5].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [5] [5] Anonymous. 2022. Seen on screen: The importance of disability representation . Available from https://www.nielsen.com/[Accessed30-08-2025]. -
Not checked in the free run.
- Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3
Media portrayals of people with disabilities often portray them as super-heroes overcoming insurmountable barriers.
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] -portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [34] [34] Katie Ellis, Gerard Goggin, Beth A. Haller, and Rosemary Curtis (Eds.). 2020. The Routledge Companion to Disability and Media . Routledge, New York. https: //doi.org/10.4324/9781315716008 - source
- cites [100] [100] Summer S Shelton and T Franklin Waddell. 2021. Does 'inspiration porn'inspire? How disability and challenge impact attitudinal evaluations of advertising. Journal of Current Issues & Research in Advertising 42, 3 (2021), 258-276. - source
- cites [120] [120] L. Zhang and B. Haller. 2013. Consuming Image: How Mass Media Impact the Identity of People with Disabilities. Communication Quarterly 61, 3 (2013), 319-334. doi:10.1080/01463373.2013.776988 - source
- Premisecitation§ 1 Introduction · p. 2
AI image generation models often omit or distort representations of assistive technologies.
“As a consequence, they currently depict PwD in unrealistic or dehumanizing ways [48, 77], and often omit or distort representations of assistive technologies [77].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [77] [77] Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K. Kane, and Cynthia L. Bennett. 2024. 'They only care to show us the wheelchair': disability representation in text-to-image AI models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 288, 23 pages. doi:10.1145/3613904.3642166 - source
- Interpretationreasoning§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
The limitations of current image generation models in representing people with disabilities stem from the data pipelines and practices typically used to train them.
“The limitations of current image generation models in representing PwD stem from the data pipelines and practices typically used to train them.”
Quote in the paperSubstantial claim - Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3
Media portrayals of people with disabilities often portray them as objects of pity, charity, or humor.
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] -portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [49] [49] Tina Goethals, Dimitri Mortelmans, Hilde Van den Bulck, Willem Van den Heurck, and Geert Van Hove and. 2022. I am not your metaphor: frames and counter-frames in the representation of disability. Disability & Society 37, 5 (2022), 746-764. arXiv:https://doi.org/10.1080/09687599.2020.1836478 doi:10. 1080/09687599.2020.1836478 - source
- cites [88] [88] Erin Pritchard. 2024. Let's Tackle Representations of Dwarfism in the Media . https://www.hope.ac.uk/news/allnews/lets-tackle-representations-ofdwarfism-in-the-media.html Liverpool Hope University News. - source
- Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3
Mack et al. found a recurring reliance on reductive archetypes in image generation models, portraying people with disabilities as sad, lonely, inactive, or incapable.
“Mack et al. [77] conducted focus groups examining how PwD were represented in image generation models. Their findings revealed a recurring reliance on reductive archetypes -portraying PwD as sad, lonely, inactive, or incapable.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [77] [77] Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K. Kane, and Cynthia L. Bennett. 2024. 'They only care to show us the wheelchair': disability representation in text-to-image AI models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 288, 23 pages. doi:10.1145/3613904.3642166 - source
- Study design and conductown data§ 4.2 Curating the Community Library · p. 12
Although the Community Library structure proposed five themes with four sub-themes (~20 images each), the authors were careful not to over-enforce but to balance this with communities' representation goals and feasibility constraints.
“Although the Community Library structure proposed five themes with four sub-themes ( ∼ 20 images each), we were careful to not over-enforce but to balance this with communities' representation goals and feasibility constraints.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ 3.1 Disability Communities & Project Leads · p. 4
The authors work with disability advocacy organizations as proxies for bounded disability communities.
“We work with disability advocacy organizations as proxies for bounded disability communities.”
Quote in the paperSubstantial claim - Interpretationown data§ 4.1 Defining Representation · p. 11
The Magazine metaphor-foregrounding how others may perceive a community through selected imagery-encouraged project leads to critically reflect on 'external' perceptions, prompting more intentional, nuanced choices in image selection and presentation.
“This illustrates how the Magazine metaphor -that foregrounds how others may come to perceive a community through selected imagery -encouraged project leads to critically reflect on 'external' perceptions of their communities, prompting more intentional, nuanced choices in image selection and presentation .”
Quote in the paperSubstantial claim - Interpretationown data§ 4.2 Curating the Community Library · p. 11
Data curation enabled project leads to broaden and clarify their community and representation definition.
“This illustrates how data curation enabled project leads to broaden and clarify their community and representation definition.”
Quote in the paperNot a substantial claim - Interpretationown data§ 4.1 Defining Representation · p. 10
Familiar visual norms can constrain representation.
“This highlights how familiar visual norms can constrain representation , and demonstrates how the Pinboard activity encouraged project leads to imagine aspirational representations that expand what is typically seen or considered possible today .”
Quote in the paperSubstantial claim - Interpretationreasoning§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 15
The Community Library imposes a 'thematic structure' and requirements for a 'balance' in data across thematic groups, which could inadvertently risk pigeonholing communities in how they may wish to express themselves or categorize information.
“One key structure for organizing community inputs is the Community Library. Through its design, it imposes a 'thematic structure' and requirements for a 'balance' in data across thematic groups, which could inadvertently risk pigeonholing communities' in the ways they may wish to express themselves or categorize information.”
Quote in the paperSubstantial claim - Interpretationown data§ 4.3 Evaluating AI · p. 15
Project leads applied their specific domain expertise to identify subtle nuances and make careful judgments about the images.
“This shows how project leads applied their specific domain expertise to identify subtle nuances and make careful judgments about the images , by paying close attention to details of the stepping stool that did not align with their own experiences or understandings of what is considered appropriate or logical in this situation.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 15
By choosing these structures, the authors stirred the direction of how communities approached defining representation.
“However, we acknowledge that by choosing these structures, we stirred the direction of how communities approached defining representation.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 16
A community-centric approach to creating image descriptions and prompts, focusing less on 'what' an image entails and instead asking 'why' the image meaningfully reflects the community, can bring a greater emphasis on human 'values' into data annotation practices.
“Firstly, we took a community-centric approach to the creation of 'image descriptions' and the 'prompts' we derived from it, by focusing less on 'what' an image entails and instead asking 'why' the image meaningfully reflects the community. Through this we exemplify how we can bring a greater emphasis on human 'values' into data annotation practices .”
Quote in the paperSubstantial claim - Interpretationreasoning§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 15
The Community Library structure provides a useful boundary for supporting communities in prioritizing their representation goals whilst making data requirements for AI more concrete.
“As such, the Community Library structure provides a useful boundary for supporting communities in prioritizing their representation goals whilst making data requirements for AI more concrete .”
Quote in the paperSubstantial claim - Interpretationreasoning§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15
The contextualized, interpretative nature of definitions by each community extends the scope of relevant criteria beyond the accuracy of observable aspects of the physical world.
“All this highlights the contextualized, interpretative nature of definitions by each community that clearly extend the scope of relevant criteria to consider beyond the accuracy of observable aspects of the physical world (e.g., is the AT in itself correctly rendered).”
Quote in the paperSubstantial claim - Interpretationown data§ 4.3 Evaluating AI · p. 15
There is a broader challenge in image generation models to interpret the functional, embodied realities of assistive technologies beyond more simplistic assessments of objective features of the physical world.
“This demonstrates a broader challenge in image generation models to interpret the functional, embodied realities of assistive technologies (e.g., is this the right stepping stool for this community) beyond often more simplistic assessments of objective features of the physical world (e.g., whether a stepping stool was generated or not).”
Quote in the paperSubstantial claim - Interpretationown data§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15
Material engagement with images, through data curation, manipulation, and annotation, offers an additional pathway for communities to refine their representation definition and embed these nuanced understandings directly within data.
“These findings show that material engagement with images-through data curation, manipulation, and annotation offers an additional pathway for communities to refine their representation definition and embed these nuanced understandings directly within data .”
Quote in the paperSubstantial claim - Study design and conductreasoning§ 5.3 Human-Centred AI Data Work · p. 16
In their work, the authors specifically chose to minimize community exposure to overly negative AI outputs, for example by filtering out clearly offensive images for their AI evaluation task.
“In our work, we therefore specifically chose to minimize community exposure to overly negative AI outputs , for example, by filtering out clearly offensive images for our AI evaluation task.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 5.3 Human-Centred AI Data Work · p. 16
The positive approach adopted throughout all system scaffolds, asking for uploads of 'good' images or prompting reflection on why an image was a 'good' representation, centers the community's lived experiences and aspirations for more meaningful representation rather than requiring them to deeply engage with the limitations and failings of image models.
“This strategy centers the community's lived experiences and aspirations for more meaningful representation rather than requiring them to deeply engage with the limitations and failings of image models .”
Quote in the paperSubstantial claim
- May combine two claims. 60
- Classified as probably not a substantial claim. 65
May combine two claims.
Premisecitation§ 1 Introduction · p. 2Current data collection processes prioritize scale over inclusion, relying on web-scraping and synthetic pipelines that reflect developer biases and the uneven availability of online content.
“Current data collection processes prioritize scale over inclusion, relying on web-scraping (e.g., [40, 97]) and synthetic pipelines that reflect developer biases and the uneven availability of online content [13, 14, 57, 96].”
Quote in the paperSubstantial claim3 of 6 sources checked- cites [40] [40] Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. 2023. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems 36 (2023), 2709227112. - source
The Collection Process section describes downloading data from the public internet and parsing Common Crawl, while Q42 says COMMONPOOL reflects societal biases present in that internet data. The paper does not establish that collection processes prioritize scale over inclusion or that synthetic pipelines reflect developer biases; the cited passages also do not assess uneven online availability. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [97] [97] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems 35 (2022), 25278-25294. - source
- cites [13] [13] Abeba Birhane, Sepehr Dehdashtian, Vinay Prabhu, and Vishnu Boddeti. 2024. The dark side of dataset scaling: Evaluating racial classification in multimodal models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 1229-1244. - source
The Introduction (p. 3) describes dataset creation through internet scraping and Common Crawl, while the Discussion (p. 13) says scale is prized for training and describes aggressive crawl-and-scrape strategies. These passages support the scale and web-scraping portions, but they do not establish the claim about synthetic pipelines, developer biases, or uneven availability of online content. (LLM, conf=0.82)
Experimental — accuracy not yet measured - cites [14] [14] Abeba Birhane and Vinay Uday Prabhu. 2021. Large image datasets: A pyrrhic win for computer vision?. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) . IEEE, 1536-1546. - source
- cites [57] [57] Rachel Hong, William Agnew, Tadayoshi Kohno, and Jamie Morgenstern. 2024. Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp. In Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization . 1-17. - source
The Introduction (p. 1) says datasets are scraped from the web to reach a scale of billions, and the Abstract (p. 1) reports bias and amplified underrepresentation in the resulting filtered data. But this paper studies filtering of web-scraped data, not synthetic data pipelines; it does not support that part of the claim. (LLM, conf=0.87)
Experimental — accuracy not yet measured - cites [96] [96] Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. 'Everyone wants to do the model work, not the data work': Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-15. - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3Some people embrace disability as a core identity depending on person and social context.
“Furthermore, disclosure varies by person and context: some people, and depending on the social context, embrace disability as a core identity, while others generally or temporarily minimize it to 'present a capable self' [118].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [118] [118] Kexin Zhang, Elmira Deldari, Zhicong Lu, Yaxing Yao, and Yuhang Zhao. 2022. 'It's Just Part of Me:' Understanding Avatar Diversity and Self-presentation of People with Disabilities in Social Virtual Reality. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (Athens, Greece) (ASSETS '22) . Association for Computing Machinery, New York, NY, USA, Article 4, 16 pages. doi:10.1145/3517428.3544829 - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3The literature suggests flexibility, providing a range of potential depictions.
“As such, the literature suggests flexibility, providing a range of potential depictions [76, 118, 119].”
Quote in the paperNot a substantial claim3 of 3 sources checked- cites [76] [76] Kelly Mack, Rai Ching Ling Hsu, Andrés Monroy-Hernández, Brian A. Smith, and Fannie Liu. 2023. Towards Inclusive Avatars: Disability Representation in Avatar Platforms. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23) . Association for Computing Machinery, New York, NY, USA, Article 607, 13 pages. doi:10.1145/3544548. 3581481 - source
- cites [118] [118] Kexin Zhang, Elmira Deldari, Zhicong Lu, Yaxing Yao, and Yuhang Zhao. 2022. 'It's Just Part of Me:' Understanding Avatar Diversity and Self-presentation of People with Disabilities in Social Virtual Reality. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (Athens, Greece) (ASSETS '22) . Association for Computing Machinery, New York, NY, USA, Article 4, 16 pages. doi:10.1145/3517428.3544829 - source
- cites [119] [119] Kexin Zhang, Edward Glenn Scott Spencer, Abijith Manikandan, Andric Li, Ang Li, Yaxing Yao, and Yuhang Zhao. 2025. Inclusive Avatar Guidelines for People with Disabilities: Supporting Disability Representation in Social Virtual Reality. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25) . Association for Computing Machinery, New York, NY, USA, Article 560, 26 pages. doi:10.1145/3706598.3714230 - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3Some people generally or temporarily minimize disability to 'present a capable self'.
“Furthermore, disclosure varies by person and context: some people, and depending on the social context, embrace disability as a core identity, while others generally or temporarily minimize it to 'present a capable self' [118].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [118] [118] Kexin Zhang, Elmira Deldari, Zhicong Lu, Yaxing Yao, and Yuhang Zhao. 2022. 'It's Just Part of Me:' Understanding Avatar Diversity and Self-presentation of People with Disabilities in Social Virtual Reality. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (Athens, Greece) (ASSETS '22) . Association for Computing Machinery, New York, NY, USA, Article 4, 16 pages. doi:10.1145/3517428.3544829 - source
Classified as probably not a substantial claim.
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3Existing research found that positive representation requires empowering users to control how and when they disclose their disability.
“Existing research found that positive representation requires empowering users to control how and when they disclose their disability -whether by showing physical differences, assistive technologies (ATs), or through behaviors and symbols like movement patterns, sign language, or cultural icons [76, 119].”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [76] [76] Kelly Mack, Rai Ching Ling Hsu, Andrés Monroy-Hernández, Brian A. Smith, and Fannie Liu. 2023. Towards Inclusive Avatars: Disability Representation in Avatar Platforms. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23) . Association for Computing Machinery, New York, NY, USA, Article 607, 13 pages. doi:10.1145/3544548. 3581481 - source
- cites [119] [119] Kexin Zhang, Edward Glenn Scott Spencer, Abijith Manikandan, Andric Li, Ang Li, Yaxing Yao, and Yuhang Zhao. 2025. Inclusive Avatar Guidelines for People with Disabilities: Supporting Disability Representation in Social Virtual Reality. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25) . Association for Computing Machinery, New York, NY, USA, Article 560, 26 pages. doi:10.1145/3706598.3714230 - source
Classified as probably not a substantial claim.
Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3Recent calls in generative AI evaluation frameworks to separate the systematization of the concept to be measured and the operationalization of the measure technically further emphasizes the need to articulate what is evaluated before determining how to evaluate it.
“Recent calls -in generative AI evaluation frameworks [115] -to separate the systematization of the concept to be measured, and the operationalization of the measure technically,further emphasizes the need to articulate what is evaluated before determining how to evaluate it.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [115] [115] Hanna Wallach, Meera Desai, A Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alexandra Chouldechova, Emily Corvi, P Alex Dow, et al. 2025. Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge. arXiv preprint arXiv:2502.00561 (2025). - source
May combine two claims.
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3People with disabilities are frequently absent or negatively stereotyped in media.
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] -portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
Quote in the paperSubstantial claim1 of 4 sources checked- cites [34] [34] Katie Ellis, Gerard Goggin, Beth A. Haller, and Rosemary Curtis (Eds.). 2020. The Routledge Companion to Disability and Media . Routledge, New York. https: //doi.org/10.4324/9781315716008 - source
- cites [49] [49] Tina Goethals, Dimitri Mortelmans, Hilde Van den Bulck, Willem Van den Heurck, and Geert Van Hove and. 2022. I am not your metaphor: frames and counter-frames in the representation of disability. Disability & Society 37, 5 (2022), 746-764. arXiv:https://doi.org/10.1080/09687599.2020.1836478 doi:10. 1080/09687599.2020.1836478 - source
- cites [103] [103] Jonatan Södergren and Niklas Vallström. 2023. Disability in influencer marketing: a complex model of disability representation. Journal of Marketing Management 39, 11-12 (2023), 1012-1042. - source
The Conclusion says market-mediated representations often evoke pity or ‘inspiration porn’ and depict people with disabilities as victims or superhuman, supporting the negative-stereotyping part. It does not establish that people with disabilities are frequently absent from media; it says they are becoming more visible in advertising and mainstream media. (LLM, conf=0.83)
Experimental — accuracy not yet measured - cites [120] [120] L. Zhang and B. Haller. 2013. Consuming Image: How Mass Media Impact the Identity of People with Disabilities. Communication Quarterly 61, 3 (2013), 319-334. doi:10.1080/01463373.2013.776988 - source
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3A particularly promising evaluation approach is fine-tuning MLLM 'judges' on context-aware or community-sensitive data.
“A particularly promising evaluation approach is fine-tuning MLLM 'judges' on context-aware or community-sensitive data [75, 111].”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites [75] [75] Zi-Ao Ma, Tian Lan, Rong-Cheng Tu, Shu-Hang Liu, Heyan Huang, Zhijing Wu, Chen Xu, and Xian-Ling Mao. 2025. T2I-Eval-R1: Reinforcement LearningDriven Reasoning for Interpretable Text-to-Image Evaluation. arXiv preprint arXiv:2505.17897 (2025). - source
The Introduction describes fine-tuning MLLMs as text-to-image evaluators, and Section 3.3 describes training them on image-quality scores and human preferences. Neither section identifies the training data as context-aware or community-sensitive, so that qualification is not supported by this source. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [111] [111] Rong-Cheng Tu, Zi-Ao Ma, Tian Lan, Yuehao Zhao, Heyan Huang, and XianLing Mao. 2024. Automatic evaluation for text-to-image generation: Taskdecomposed framework, distilled training, and meta-evaluation benchmark. arXiv preprint arXiv:2411.15488 (2024). - source
The Fine-tuning Settings and §5.3.1 describe fine-tuning MLLMs as automatic evaluators and report improved evaluation performance. The Text-to-image Generation section describes training on paired prompts and images, which supports task-specific context, but our copy does not describe community-sensitive training data or establish that tailoring data to communities improves evaluation. (LLM, conf=0.82)
Experimental — accuracy not yet measured
May combine two claims.
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4Data is not only relational, but also shaped by its creation context, whereby datasets reflect the worldviews and beliefs of their creators who decide what deserves capturing and how information is classified.
“This requires recognizing that data is not only relational, but also shaped by its creation context, whereby datasets reflect the worldviews and beliefs of their creators who decide what deserves capturing and how information is classified [17, 53].”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [17] [17] Geoffrey C Bowker and Susan Leigh Star. 2000. Sorting things out: Classification and its consequences . MIT press. - source
- cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4Where humans annotate data for AI image evaluation, their role commonly involves identifying objects or features in images, image comparisons, or gamified approaches.
“Where humans annotate data for AI image evaluation, their role commonly involves: identifying objects or features in images [52]; image comparisons to indicate similarities or preferences across two or more samples (e.g., [52, 85]); or gamified approaches (e.g., describing artifacts characteristics for another annotator to guess [99]).”
Quote in the paperNot a substantial claim2 of 3 sources checked- cites [52] [52] Melissa Hall, Samuel J Bell, Candace Ross, Adina Williams, Michal Drozdzal, and Adriana Romero Soriano. 2024. Towards geographic inclusion in the evaluation of text-to-image models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 585-601. - source
- cites [85] [85] Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkilä, and Shin'ichi Satoh. 2023. Toward verifiable and reproducible human evaluation for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 14277-14286. - source
The annotation-interface passage (Section 2; page unavailable) and Figure A describe annotators rating images for fidelity and caption alignment, not identifying objects or features. I searched for object-identification tasks and annotator comparisons of image samples; the paper’s pairwise comparisons (Section 3; page unavailable) are statistical tests of ratings, not comparisons performed by annotators. The gamified approach is cited to a different source in the citing sentence. (LLM, conf=0.72)
Experimental — accuracy not yet measured - cites [99] [99] Agrima Seth, Sanchit Ahuja, Kalika Bali, and Sunayana Sitaram. 2024. Dosa: A dataset of social artifacts from different indian geographical subcultures. arXiv preprint arXiv:2403.14651 (2024). - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3Recent social science work has proposed richer evaluation categories including incorrectness, missingness, specificity, coherence, and connotation.
“Taking a different perspective, recent social science work [89] has proposed richer evaluation categories that can give more nuance to assessments of cultural images, including: incorrectness, missingness, specificity, coherence, and connotation.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [89] [89] Rida Qadri, Mark Diaz, Ding Wang, and Michael Madaio. 2025. The case for" thick evaluations" of cultural representation in ai. arXiv preprint arXiv:2503.19075 (2025). - source
Classified as probably not a substantial claim.
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 3Human-centered AI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users and other impacted stakeholders.
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
Quote in the paperSubstantial claim2 of 7 sources checked- cites [4] [4] Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems . 1-13. - source
- cites [8] [8] Eric PS Baumer. 2017. Toward human-centered algorithm design. Big Data & Society 4, 2 (2017), 2053951717718854. - source
- cites [16] [16] Elizabeth Bondi, Lily Xu, Diana Acosta-Navas, and Jackson A Killian. 2021. Envisioning communities: a participatory approach towards AI for social good. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society . 425-436. - source
- cites [99] [99] Agrima Seth, Sanchit Ahuja, Kalika Bali, and Sunayana Sitaram. 2024. Dosa: A dataset of social artifacts from different indian geographical subcultures. arXiv preprint arXiv:2403.14651 (2024). - source
- cites [101] [101] Ben Shneiderman. 2021. Human-centered AI: A new synthesis. In IFIP Conference on Human-Computer Interaction . Springer, 3-8. - source
- cites [104] [104] Hariharan Subramonyam, Colleen Seifert, and MI Eytan Adar. 2021. How can human-centered design shape data-centric AI. In Proceedings of NeurIPS Data-Centric AI Workshop . Curran Associates New York, NY, USA, 3. -
Not checked in the free run.
- cites [110] [110] Anja Thieme, Aditya Nori, Marzyeh Ghassemi, Rishi Bommasani, Tariq Osman Andersen, and Ewa Luger. 2023. Foundation models in healthcare: Opportunities, risks & strategies forward. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems . 1-4. - source
Section 2.3 criticizes developing AI without understanding intended users’ needs and calls for attention to people who may benefit or be harmed. It does not specifically call for actively involving those people in AI design or say that such involvement reflects their values and preferences. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4In disability contexts, key datasets include ORBIT, StammerTalk, ASL Citizen, and the grassroots Disabled And Here stock image collection.
“In disability contexts, key datasets include ORBIT [80] -videos and images of objects recorded by people who are blind or low-vision; StammerTalk [74] and ASL Citizen [32] for speech and language impairments; and the grassroots Disabled And Here stock image collection celebrating disabled Black, Indigenous, People of Colour (BIPOC) [6].”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [80] [80] Daniela Massiceti, Luisa Zintgraf, John Bronskill, Lida Theodorou, Matthew Tobias Harris, Edward Cutrell, Cecily Morrison, Katja Hofmann, and Simone Stumpf. 2021. Orbit: A real-world few-shot dataset for teachable object recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 10818-10828. - source
- cites [74] [74] Qisheng Li and Shaomei Wu. 2024. "I Want to Publicize My Stutter": Communityled Collection and Curation of Chinese Stuttered Speech Data. Proc. ACM Hum.-Comput. Interact. 8, CSCW2, Article 475 (Nov. 2024), 27 pages. doi:10. 1145/3687014 -
Not checked in the free run.
- cites [32] [32] Aashaka Desai, Lauren Berger, Fyodor Minakov, Nessa Milano, Chinmay Singh, Kriston Pumphrey, Richard Ladner, Hal Daumé III, Alex X Lu, Naomi Caselli, et al. 2023. ASL citizen: a community-sourced dataset for advancing isolated sign language recognition. Advances in Neural Information Processing Systems 36 (2023), 76893-76907. - source
The Introduction identifies ASL Citizen as a crowdsourced dataset of isolated ASL signs, supporting its inclusion as a dataset in a disability-related list. But it describes sign language as central to Deaf culture and identity, not as a dataset for speech or language impairments; that classification is unsupported for ASL Citizen. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4Examples of data stewardship include data trusts or cooperatives, which use legal frameworks to formalize collective data management such as data commons or community archives.
“Examples include data trusts or cooperatives, which use legal frameworks to formalize collective data management such as data commons [30, 112] or community archives.”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [30] [30] Sylvie Delacroix and Neil D Lawrence. 2019. Bottom-up data trusts: Disturbing the 'one size fits all'approach to data governance. International data privacy law 9, 4 (2019), 236-252. - source
In “The Trust structure as a way of taking into account the vulnerabilities at stake” (page unavailable), the paper describes a legal Trust that pools data rights and requires a trustee to manage them for beneficiaries. “Information fiduciaries” (page unavailable) says some data Trusts may resemble a collective or cooperative. The passages support the data-trust and collective-management parts, but do not identify data commons or community archives as examples. (LLM, conf=0.84)
Experimental — accuracy not yet measured - cites [112] [112] Jonathan Van Geuns, Ana Brandusescu, and Mozilla Insights. 2020. What Does it Mean? Shifting Power Through Data Governance. Mozilla Foundation 2020)< https://foundation. mozilla. org/en/data-futureslab/data-forempowerment/shifting-power-through-data-governance/> accessed 8 September 2025 (2020). -
Not checked in the free run.
Classified as probably not a substantial claim.
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4AI data stewardship applies this approach to dataset collection for training AI models, especially when these have knowledge gaps about marginalized communities.
“AI data stewardship applies this approach to dataset collection for training AI models, especially when these have knowledge gaps about marginalized communities (cf. [20, 59]).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [20] [20] Stephanie Russo Carroll, Desi Rodriguez-Lonebear, and Andrew Martinez. 2019. Indigenous data governance: strategies from United States native nations. Data science journal 18 (2019), 31. [doi:10.1145/3649883](https://doi.org/10.1145/3649883) - source
The “Governance of Data” section supports stewardship of Indigenous data and warns that excluding Indigenous peoples from large datasets can make them invisible, while ungoverned inclusion can reproduce bias. It does not discuss collecting datasets to train AI models or establish that AI data stewardship applies this approach to marginalized communities generally. (LLM, conf=0.85)
Experimental — accuracy not yet measured - cites [59] [59] Maui Hudson, Stephanie Russo Carroll, Jane Anderson, Darrah Blackwater, Felina M Cordova-Marks, Jewel Cummins, Dominique David-Chavez, Adam 21. Fernandez, Ibrahim Garba, Danielle Hiraldo, et al. 2023. Indigenous Peoples' rights in data: a contribution toward Indigenous Research Sovereignty. Frontiers in Research Metrics and Analytics 8 (2023), 1173805. - source
The “Indigenous Data Sovereignty and research” section calls for Indigenous participation in data governance across the data lifecycle and explicitly identifies dataset diversity for AI as important. The “Right to self-determination” section describes Indigenous-led collection to fill community data gaps. The paper does not specifically discuss collecting datasets to train AI models or AI models’ knowledge gaps about marginalized communities generally. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ COMMUNITY-LED ACTIVITIES · p. 8Ratings serve as a core input to model alignment pipelines, underpinning approaches like reinforcement learning from human feedback (RLHF), and forming the basis of large-scale evaluation platforms such as LM-Arena.
“Ratings also serve as a core input to model alignment pipelines, underpinning approaches like reinforcement learning from human feedback (RLHF) [26], and forming the basis of large-scale evaluation platforms such as LM-Arena [23].”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [26] [26] Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems 30 (2017). - source
The Introduction (pp. 1–2) says the authors learn a reward function from human feedback and use it to train a reinforcement-learning agent, supporting the RLHF portion attributed to this source. But their input is pairwise comparisons of behavior, explicitly “rather than” absolute numerical ratings. The LM-Arena clause has a separate citation and is not judged here. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [23] [23] Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. 2024. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning . -
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 16Variation in how annotators subjectively interpret and follow concrete definitions has also been noted in other image generation data annotation work.
“Variation in how annotators subjectively interpret and follow concrete definitions has also been noted in other image generation data annotation work [52].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [52] [52] Melissa Hall, Samuel J Bell, Candace Ross, Adina Williams, Michal Drozdzal, and Adriana Romero Soriano. 2024. Towards geographic inclusion in the evaluation of text-to-image models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 585-601. - source
May combine two claims.
Premisecitation§ 5.3 Human-Centred AI Data Work · p. 16Existing research predominantly focuses on understanding, defining or evaluating representational 'harms' and often uses poor AI-generated images to elicit feedback from marginalized communities.
“Our approach to defining and evaluating 'good' representation differs from existing research that predominantly focuses on understanding, defining or evaluating representational 'harms' [24, 28, 38, 44, 67, 77, 91, 116] and often uses poor AI-generated images to elicit feedback from marginalized communities (e.g., [65, 77, 79]).”
Quote in the paperSubstantial claim6 of 10 sources checked- cites [24] [24] Jennifer Chien and David Danks. 2024. Beyond behaviorist representational harms: A plan for measurement and mitigation. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 933-946. - source
The Introduction and Section 2 examine existing definitions of representational harms, supporting the citation’s use as an example of harm-focused research. The paper does not support the further claim that it uses poor AI-generated images to elicit feedback from marginalized communities; Section 4.1 recommends engaging affected communities but describes no such image-feedback exercise. Page numbers are unavailable in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [28] [28] Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alexandra Chouldechova, P Alex Dow, Jean Garcia-Gathright, Nicholas Pangakis, Emily Sheng, Dan Vann, et al. 2025. Taxonomizing Representational Harms using Speech Act Theory. arXiv preprint arXiv:2504.00928 (2025). - source
The Introduction and Conclusion show that this paper defines and examines representational harms, supporting the first part of the attribution. The Limitations section says it focuses on language generation rather than images; the cited paper does not support the claim that it uses poor AI-generated images to elicit feedback from marginalized communities. (LLM, conf=0.90)
Experimental — accuracy not yet measured - cites [38] [38] Vinitha Gadiraju, Shaun Kane, Sunipa Dev, Alex Taylor, Ding Wang, Remi Denton, and Robin Brewer. 2023. " I wouldn't say offensive but... ": DisabilityCentered Perspectives on Large Language Models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency . 205-216. -
Not checked in the free run.
- cites [44] [44] Sourojit Ghosh, Nina Lutz, and Aylin Caliskan. 2024. 'I Don't See Myself Represented Here at All': User Experiences of Stable Diffusion Outputs Containing Representational Harms across Gender Identities and Nationalities. In Proceedings of the AAAI/ACM conference on AI, ethics, and society , Vol. 7. 463-475. - source
- cites [67] [67] Jared Katzman, Angelina Wang, Morgan Scheuerman, Su Lin Blodgett, Kristen Laird, Hanna Wallach, and Solon Barocas. 2023. Taxonomizing and measuring representational harms: A look at image tagging. In Proceedings of the AAAI Conference on artificial intelligence , Vol. 37. 14277-14285. - source
The Introduction (page unavailable) says the paper identifies types of representational harm and evaluates approaches to measuring them, supporting the first part of the claim. The Computational Measurement Approaches section (page unavailable) mentions GAN-generated faces used to test image-tagging systems, but does not describe using poor AI-generated images to elicit feedback from marginalized communities. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [77] [77] Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K. Kane, and Cynthia L. Bennett. 2024. 'They only care to show us the wheelchair': disability representation in text-to-image AI models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 288, 23 pages. doi:10.1145/3613904.3642166 - source
- cites [91] [91] Rida Qadri, Renee Shelby, Cynthia L. Bennett, and Remi Denton. 2023. AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 506-517. doi:10.1145/3593013. 3594016 - source
- cites [116] [116] Angelina Wang, Solon Barocas, Kristen Laird, and Hanna Wallach. 2022. Measuring representational harms in image captioning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency . 324-335. - source
The Introduction and §5.1 describe the paper’s aim as developing techniques to measure representational harms in image captioning, supporting its inclusion among research focused on harms. The methods use system-generated captions, while §5.1 proposes incorporating affected people’s lived experiences as future work. The paper does not support the claim that it used poor AI-generated images to elicit feedback from marginalized communities. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [65] [65] Nari Johnson, Hamna ., Deepthi Sudharsan, Theo Holroyd, Samantha Dalal, Siobhan Mackenzie Hall, Jennifer Wortman Vaughan, Daniela Massiceti, and Cecily Morrison. 2025. Position: To Make Text-to-Image Models that Work for Marginalized Communities, We Need New Measurement Practices for the Long Tail. (July 2025). https://www.microsoft.com/en-us/research/wpcontent/uploads/2025/07/longtail.pdf. -
Not checked in the free run.
- cites [79] [79] Daniela Massiceti, Camilla Longden, Agnieszka Słowik, Samuel Wills, Martin Grayson, and Cecily Morrison. 2024. Explaining CLIP's performance disparities on data from blind/low vision users. arXiv:2311.17315 [cs.CV] https://arxiv.org/ abs/2311.17315 - source
Section 5.2 reports that DALL-E2 often generated incorrect images of disability-related objects, supporting the point about poor AI-generated images. But the images were evaluated by three annotators; our copy does not say they were used to elicit feedback from marginalized communities. The paper also studies performance disparities rather than establishing that existing research predominantly focuses on representational harms. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Premisecitation§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 16Recent works argue against treatments of representation as a static concept that can be evaluated objectively.
“This echoes recent works [10, 68, 89, 90] that argue against treatments of representation as a static concept that can be evaluated objectively.”
Quote in the paperSubstantial claim3 of 4 sources checked- cites [10] [10] Cynthia L Bennett, Shaun K Kane, and Christina N Harrington. 2025. Toward Community-Led Evaluations of Text-to-Image AI Representations of Disability, Health, and Accessibility. In Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization . 256-270. - source
- cites [68] [68] Sarah Kiden, Oriane Peter, Gisela Reyes-Cruz, Maira Klyshbekova, Sena Choi, Aislinn Gomez Bergin, Maria Waheed, Damian Eke, Tayyaba Azim, Sarvapali Ramchurn, et al. 2025. Back to the Communities: A Mixed-Methods and Community-Driven Evaluation of Cultural Sensitivity in Text-to-Image Models. arXiv preprint arXiv:2510.27361 (2025). - source
- cites [89] [89] Rida Qadri, Mark Diaz, Ding Wang, and Michael Madaio. 2025. The case for" thick evaluations" of cultural representation in ai. arXiv preprint arXiv:2503.19075 (2025). - source
- cites [90] [90] Rida Qadri, Michael Madaio, and Mary L. Gray. 2025. Confusing the Map for the Territory. Commun. ACM (Sept. 2025), 4 pages. doi:10.1145/3731655 Online First. - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 5.3 Human-Centred AI Data Work · p. 16Other trends in AI evaluation include red-teaming, whereby image generation models or systems are deliberately tested with adversarial prompts to uncover vulnerabilities and bias, surfacing offensive or harmful content for error detection or removal.
“Other trends in AI evaluation include red-teaming, whereby image generation models or systems are deliberately tested with adversarial prompts to uncover their 'vulnerabilities' and 'bias'; surfacing offensive or harmful content for purposes of error detection, or removal [46].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [46] [46] Tarleton Gillespie, Ryland Shaw, Mary L Gray, and Jina Suh. 2024. AI redteaming is a sociotechnical challenge: on values, labor, and harms. arXiv preprint arXiv:2412.09751 (2024). - source
May combine two claims.
Premisecitation§ 5.3 Human-Centred AI Data Work · p. 17In more community-centric data collection or AI evaluation work, active efforts are made by researchers to solicit broad participation or allow community members to self-select their participation, albeit with some limitations.
“In more community-centric data collection or AI evaluation work, active efforts are made by researchers to solicit broad participation (e.g., across geographical regions [52, 68, 91] and through local research partners [89]), or allow community members to selfselect their participation, albeit with some limitations (e.g., adults, higher-education levels or topic expertise, English-language ability”
Quote in the paperNot a substantial claim4 of 6 sources checked- cites [52] [52] Melissa Hall, Samuel J Bell, Candace Ross, Adina Williams, Michal Drozdzal, and Adriana Romero Soriano. 2024. Towards geographic inclusion in the evaluation of text-to-image models. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency . 585-601. - source
- cites [68] [68] Sarah Kiden, Oriane Peter, Gisela Reyes-Cruz, Maira Klyshbekova, Sena Choi, Aislinn Gomez Bergin, Maria Waheed, Damian Eke, Tayyaba Azim, Sarvapali Ramchurn, et al. 2025. Back to the Communities: A Mixed-Methods and Community-Driven Evaluation of Cultural Sensitivity in Text-to-Image Models. arXiv preprint arXiv:2510.27361 (2025). - source
- cites [91] [91] Rida Qadri, Renee Shelby, Cynthia L. Bennett, and Remi Denton. 2023. AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 506-517. doi:10.1145/3593013. 3594016 - source
- cites [89] [89] Rida Qadri, Mark Diaz, Ding Wang, and Michael Madaio. 2025. The case for" thick evaluations" of cultural representation in ai. arXiv preprint arXiv:2503.19075 (2025). - source
- cites [78] [78] Jabez Magomere, Shu Ishida, Tejumade Afonja, Aya Salama, Daniel Kochin, Yuehgoh Foutse, Imane Hamzaoui, Raesetje Sefala, Aisha Alaagib, Samantha Dalal, et al. 2025. The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency . 246282. - source
- cites [99] [99] Agrima Seth, Sanchit Ahuja, Kalika Bali, and Sunayana Sitaram. 2024. Dosa: A dataset of social artifacts from different indian geographical subcultures. arXiv preprint arXiv:2403.14651 (2024). - source
May combine two claims.
Findingcitation§ 5.3 Human-Centred AI Data Work · p. 17Project leads spoke local languages and had intimate knowledge of various sub-groups within their community to identify individuals or regions who may be underserved; this played a key role in accessing and connecting with more marginalized, harder-to-reach populations such as blind farmers in rural areas or school-aged children.
“Project leads spoke local languages and had intimate knowledge of various sub-groups within their community to identify individuals or regions who may be underserved. This played a key role in accessing and connecting with more marginalized, harder-toreach populations (cf. [53]) such as blind farmers in rural areas or school-aged children, who otherwise may not have known about, or understood why”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
Classified as probably not a substantial claim.
Premisecitation§ 5.3 Human-Centred AI Data Work · p. 17Researchers have previously problematized anonymous crowdsourced data annotation approaches as being unspecific about who's worldviews and beliefs become manifest in the data.
“Researchers have previously problematized anonymous crowdsourced data annotation approaches [31, 53, 65] as being unspecific about who's worldviews and beliefs become manifest in the data [11, 17, 51, 78, 90, 91].”
Quote in the paperSubstantial claim6 of 9 sources checked- cites [31] [31] Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA) (EAAMO '23) . Association for Computing Machinery, New York, NY, USA, Article 37, 23 pages. doi:10.1145/3617694.3623261 - source
Section 4.3.1 (“Mistaken identity”; page unavailable) questions whether people selected as stand-ins can represent others’ preferences and values, and warns that selection bias can exclude marginalized community members. The Introduction mentions data labeling and annotation, but our copy does not make this critique specifically about anonymous crowdsourced annotation or say that such approaches leave annotators’ worldviews unidentified. (LLM, conf=0.79)
Experimental — accuracy not yet measured - cites [53] [53] Siobhan Mackenzie Hall, Samantha Dalal, Raesetje Sefala, Foutse Yuehgoh, Aisha Alaagib, Imane Hamzaoui, Shu Ishida, Jabez Magomere, Lauren Crais, Aya Salama, et al. 2025. The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes. arXiv preprint arXiv:2502.05961 (2025). - source
Section 2.1 says datasets carry their creators’ worldviews and beliefs, while Section 2.2 criticizes top-down crowdsourcing for excluding some perspectives. The paper does not specifically argue that anonymous annotation leaves it unclear whose worldviews and beliefs appear in the data; Section 4.3.2 describes accepting anonymous submissions while still collecting demographic information. No page numbers were provided in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [65] [65] Nari Johnson, Hamna ., Deepthi Sudharsan, Theo Holroyd, Samantha Dalal, Siobhan Mackenzie Hall, Jennifer Wortman Vaughan, Daniela Massiceti, and Cecily Morrison. 2025. Position: To Make Text-to-Image Models that Work for Marginalized Communities, We Need New Measurement Practices for the Long Tail. (July 2025). https://www.microsoft.com/en-us/research/wpcontent/uploads/2025/07/longtail.pdf. -
Not checked in the free run.
- cites [11] [11] Stevie Bergman, Nahema Marchal, John Mellor, Shakir Mohamed, Iason Gabriel, and William Isaac. 2024. STELA: a community-centred approach to norm elicitation for AI alignment. Scientific Reports 14, 1 (2024), 6616. - source
- cites [17] [17] Geoffrey C Bowker and Susan Leigh Star. 2000. Sorting things out: Classification and its consequences . MIT press. - source
- cites [51] [51] Nitesh Goyal, Ian D Kivlichan, Rachel Rosen, and Lucy Vasserman. 2022. Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1-28. - source
- cites [78] [78] Jabez Magomere, Shu Ishida, Tejumade Afonja, Aya Salama, Daniel Kochin, Yuehgoh Foutse, Imane Hamzaoui, Raesetje Sefala, Aisha Alaagib, Samantha Dalal, et al. 2025. The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency . 246282. - source
Section 5 (‘Food for thought’; page unavailable) argues that who contributes to a dataset affects its representations and that community involvement helps align bias evaluation with local values. Section 2.1 describes collecting culturally grounded contributions. The paper does not specifically characterize anonymous crowdsourced annotation as leaving annotators’ worldviews or beliefs unspecified. (LLM, conf=0.76)
Experimental — accuracy not yet measured - cites [90] [90] Rida Qadri, Michael Madaio, and Mary L. Gray. 2025. Confusing the Map for the Territory. Commun. ACM (Sept. 2025), 4 pages. doi:10.1145/3731655 Online First. - source
- cites [91] [91] Rida Qadri, Renee Shelby, Cynthia L. Bennett, and Remi Denton. 2023. AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (Chicago, IL, USA) (FAccT '23) . Association for Computing Machinery, New York, NY, USA, 506-517. doi:10.1145/3593013. 3594016 - source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ 5.3 Human-Centred AI Data Work · p. 16The inherent social nature of data production and the enjoyment from working 'together' towards a shared goal echoes findings from other community-oriented AI dataset specification and curation work.
“The inherent social nature of data production and the enjoyment from working 'together' towards a shared goal, echoes findings from other community-oriented AI dataset specification and curation work [32, 99, 107].”
Quote in the paperNot a substantial claim1 of 3 sources checked- cites [32] [32] Aashaka Desai, Lauren Berger, Fyodor Minakov, Nessa Milano, Chinmay Singh, Kriston Pumphrey, Richard Ladner, Hal Daumé III, Alex X Lu, Naomi Caselli, et al. 2023. ASL citizen: a community-sourced dataset for advancing isolated sign language recognition. Advances in Neural Information Processing Systems 36 (2023), 76893-76907. - source
Section 3.1 (Data Collection; page unavailable) describes contributors making videos for a communal dictionary, and Section 6 (Discussion and Conclusion; page unavailable) describes a participatory approach. These support the community-oriented dataset aspect, but the passages do not report that contributors enjoyed working together or experienced data production as inherently social. (LLM, conf=0.78)
Experimental — accuracy not yet measured - cites [99] [99] Agrima Seth, Sanchit Ahuja, Kalika Bali, and Sunayana Sitaram. 2024. Dosa: A dataset of social artifacts from different indian geographical subcultures. arXiv preprint arXiv:2403.14651 (2024). - source
- cites [107] [107] Mei Tan, Hansol Lee, Dakuo Wang, and Hari Subramonyam. 2024. Is a seat at the table enough? Engaging teachers and students in dataset specification for ml in education. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024), 1-32. -
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ 5.4 Limitations & Future Work · p. 17Prior work suggests that small, high quality datasets can support posttraining alignment and adaptation; more data intensive pretraining settings raise challenges around data scaling without compromising authenticity.
“While prior work suggests that small, high quality datasets can support posttraining alignment and adaptation [86, 122]; more data intensive pretraining settings raise challenges around data scaling (without compromising authenticity).”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [86] [86] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730-27744. - source
- cites [122] [122] Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. Advances in Neural Information Processing Systems 36 (2023), 5500655021. source
Classified as probably not a substantial claim. May combine two claims.
Premisecitation§ 5.3 Human-Centred AI Data Work · p. 16Researchers caution that involving marginalized communities in elaborating on their marginalization for red-teaming can become transactional, extractive and exploitative, and warn of emotional and mental health costs of this type of human labor.
“However, researchers caution that involving marginalized communities in elaborating on their marginalization for red-teaming, can become transactional, extractive and exploitative (cf. [29, 46]); and warn of the emotional and mental health costs of this type of human 'labor'.”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [29] [29] Samantha Dalal, Siobhan Mackenzie Hall, and Nari Johnson. 2024. Provocation: Who benefits from" inclusion" in Generative AI? arXiv preprint arXiv:2411.09102 (2024). - source
- cites [46] [46] Tarleton Gillespie, Ryland Shaw, Mary L Gray, and Jina Suh. 2024. AI redteaming is a sociotechnical challenge: on values, labor, and harms. arXiv preprint arXiv:2412.09751 (2024). - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingreasoning§ 4.2 Curating the Community Library · p. 12Where more images fit a sub-theme, project leads decided to add or split sub-themes.
“Where more images fit a sub-theme, project leads decided to add or split sub-themes.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 12The Global North dwarfism community chose three rather than five main themes for their library, resulting in 23-46 images in each of four sub-themes.
“For example, the Global North dwarfism community chose three rather than five main themes for their library, resulting in 23-46 images in each of four sub-themes.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.2 Curating the Community Library · p. 11Project leads described the Community Library structure as helpful in planning and organizing their image collection, enabling them to track progress and define success.
“Project leads described the Community Library structure as helpful in planning and organizing their image collection, enabling them to track progress and define success.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 4.2 Curating the Community Library · p. 11The Community Library offers a 'thematic structure' to help project leads refine initially defined higher-level representation themes into specific sub-themes.
“The Community Library offers a 'thematic structure' to help project leads refine initially defined higher-level representation themes (e.g., VI education) into specific sub-themes (e.g., blind teachers; primary students; non-curricular learning).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 11Project leads navigated complex negotiations in defining 'good' representation with their community, balancing diverse perspectives with organizational goals and making choices where consensus cannot be achieved.
“Lastly, we learned how project leads navigated complex negotiations in defining 'good' representation with their community, balancing diverse perspectives with organizational goals and making choices where consensus cannot be achieved .”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 10All three organizations included 'work' as a theme, but each interpreted it differently.
“For the Magazine, all three organizations included 'work' as a theme, but each interpreted it differently:”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.2 Curating the Community Library · p. 12Where image sourcing was difficult, the authors did not enforce any minimum numbers, meaning the final dataset of the VI community had various subthemes with fewer images (e.g., digital economy (7), farming (13), competition or tournaments (13)).
“Where image sourcing was difficult, we did not enforce any minimum numbers, meaning that the final dataset of the VI community had various subthemes with fewer images (e.g., digital economy (7), farming (13), competition or tournaments (13)).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 11To engage members meaningfully in conversations about representation in AI images, project leads needed to clearly articulate the project's purpose and address concerns.
“To engage members meaningfully in conversations about representation in AI images, project leads needed to clearly articulate the project's purpose and address concerns.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 11Data availability via donations or existing archives, and direct encounters with the lived experiences of PwD, prompted project leads to add new sub-themes.
“Furthermore, (i) data availability via donations or existing archives; and (ii) direct encounters with the lived experiences of PwD prompted project leads to add new sub-themes.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingreasoning§ 4.2 Curating the Community Library · p. 11This structure guides the curation of their community's 400-image dataset and serves as foundation for image annotation.
“This structure guides the curation of their community's 400-image dataset ; and serves as foundation for image annotation .”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 11Each project lead independently mobilized their community and designed their own activities.
“Each project lead independently mobilized their community and designed their own activities (see Table 2).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 11They curated their datasets via member donations, targeted image capture, and existing archives.
“They curated their datasets via member donations (All), targeted image capture (All), and existing archives (P2, P3).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 11These methods shaped their datasets: donations yielded more individual portraits, while in-person efforts produced more group pictures.
“These methods shaped their datasets: donations (P2, P3) yielded more individual portraits (P2, P3), while in-person efforts produced more group pictures (P1).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Defining Representation · p. 10P3 notes the challenge of representing ∼400 different dwarfism types.
“Here, P3 notes the challenge of representing ∼ 400 different dwarfism types.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Defining Representation · p. 10For P1, the theme of education for people with VI is closely entwined with, and 'powered by digital assistive devices'.
“For example, for P1, the theme of education for people with VI is closely entwined with, and 'powered by digital assistive devices'.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 10For the VI organization, themes of 'education', 'assistive technology', and 'social inclusion' remained central, while 'empowerment' and 'independence' shifted towards a stronger focus on 'professional contributions to the economy' and 'sports'.
“For instance, for the VI organization, themes of 'education', 'assistive technology', and 'social inclusion' remained central, while 'empowerment' and 'independence' shifted towards a stronger focus on 'professional contributions to the economy' and 'sports'.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ 4.1 Defining Representation · p. 10P3 defined 'diversity' as a main theme and emphasized a wide range of ages, races, dwarfism types, and family configurations.
“Lastly, P3, who defined 'diversity' as a main theme for their community, emphasized a wide range of: ages, races, dwarfism types, and family configurations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ 4.1 Defining Representation · p. 10Across the three organizations, a range of higher-level themes were chosen to pursue in the Magazine activity: assistive technology (P1), independence (P1), ability (P2), diversity (P2, P3), (social) inclusion (P1, P2), empowerment (P1), pride/joy and celebration (P3), learning/education (P1, P2) and family (P3).
“Across the three organizations, a range of higherlevel themes were chosen to pursue in the Magazine activity: assistive technology (P1), independence (P1), ability (P2), diversity (P2, P3), (social) inclusion (P1, P2), empowerment (P1), pride/ joy and celebration (P3), learning/ education (P1, P2) and family (P3).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Defining Representation · p. 10For the VI organization, P1 embedded diversity within other themes to highlight the spectrum of impairment-from low vision to full blindness-and differences across urban and rural contexts.
“For the VI organization, P1 embedded diversity within other themes to highlight the spectrum of impairment -from low vision to full blindness -and differences across urban and rural contexts.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 9The Photo Pinboard activity produced nuanced accounts of what matters to a community, and how it is best articulated visually.
“This short activity produced nuanced accounts of what matters to a community, and how it is best articulated visually.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.1 Defining Representation · p. 10In the Global South dwarfism community, P2 highlighted the community's rich cultural heritage, spanning 42 tribes-expressed through unique languages, traditional clothing, and cultural artifacts.
“In the Global South dwarfism community, P2 highlighted the community's rich cultural heritage, spanning 42 tribes -expressed through unique languages, traditional clothing, and cultural artifacts.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 11For VI, community members emphasized realistic eye representation-avoiding distorted or enlarged depictions-and the correct depiction and handling of ATs.
“For dwarfism, this meant showing the right body proportions (P2); for VI, community members emphasized realistic eye representation -avoiding distorted or enlarged depictions -and the correct depiction and handling of ATs (P1).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.1 Defining Representation · p. 10The Global North dwarfism community emphasized themes of 'family', 'celebration' and 'friendship'; while the two Global South organizations featured 'education' as central to improving future opportunities for PwD, opening doors to work and income, and enabling fuller participation in life.
“The Global North dwarfism community emphasized themes of 'family', 'celebration' and 'friendship'; while the two Global South organizations featured 'education' as central to improving future opportunities for PwD, opening doors to work and income (P1), and enabling fuller participation in life (P2).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 4.1 Defining Representation · p. 10At this stage, project leads felt five thematic feature pages were sufficient to express their core representation aspirations.
“At this stage, project leads felt five thematic feature pages were sufficient to express their core representation aspirations.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingreasoning§ 4.2 Curating the Community Library · p. 12Sub-themes like 'graduation' needed fewer images to portray, while themes of 'work' required more images to reflect the diversity of potential professions and across settings.
“Furthermore, sub-themes like 'graduation' needed fewer images to portray, while themes of 'work' -broken down into 'formal', 'informal' or 'creative' industries -required more images to reflect the diversity of potential professions and across settings.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 11All organizations reported being 'very' satisfied (4/5) with their communities' representation in the Community Library in the closing survey, though they expressed interest in increasing image quality and diversity.
“All organizations reported being 'very' satisfied (4/5) with their communities' representation in the Community Library in our closing survey, though they expressed interest in increasing image quality (P2) and diversity (e.g., more children and elderly (P2), family and groups (P1), rarer disability types (P3)).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 16For creating image 'highlights', findings reveal difficulties in enabling effective annotations despite provisions of clear instructions and educational materials; project leads initially struggled to identify relevant bounding boxes and specify label details relevant for AI, leading to over-annotation of general items and inconsistencies.
“However, for the process of creating image 'highlights', our findings also reveal difficulties in enabling effective annotations despite provisions of clear instructions and educational materials. Project leads initially struggled to identify relevant bounding boxes and specify label details that would be of relevance for AI, leading to over-annotation of general items, and inconsistencies.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.3 Evaluating AI · p. 14The holistic rating score often reflected a combination of different evaluation criteria.
“Furthermore, we observed how the holistic rating score often reflected a combination of different evaluation criteria.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15Communities' definitions of representation include rich nuances, illustrated by detailed accounts of how assistive technology should be depicted as being 'handled' by a person, 'used within socially inclusive settings' and 'for STEM learning'.
“Furthermore, our findings surface rich nuances in communities' definitions. This was best illustrated in detailed accounts of how assistive technology (ATs) should be depicted as being 'handled' by a person, 'used within socially inclusive settings' and 'for STEM learning'.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.3 Evaluating AI · p. 14Scores were deducted for inaccurate rendering of assistive devices and occasionally hallucinated artifacts.
“Scores were also deducted for inaccurate rendering of assistive devices and occasionally hallucinated artifacts (e.g., disproportionate renderings of hands or feet).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.2 Curating the Community Library · p. 12P3 described how being given the agency to define this terminology felt empowering.
“P3 described how being given the agency to define this terminology felt empowering:”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.3 Evaluating AI · p. 14'Very good' representation evaluations were not only assessed based on correctness, but often associated with how realistic or natural an image depicted people, including appropriate body proportions or attire and engagement in authentic interactions within believable settings.
“We learned how 'very good' representation evaluations were not only assessed based on correctness, but often associated with how realistic or natural an image depicted people -showing the individual(s) via appropriate body proportions or attire; and as engaged in authentic interactions within believable settings (e.g., a realisticlooking marketplace).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ 4.3 Evaluating AI · p. 14In her assessment of Figure 5, P2 explained that although the demonstration of the blackboard is 'good', the image does not depict the person of short stature 'realistically' as she resembles more 'a child wearing a uniform surrounded by average height students'.
“For example, in her assessment of the image in Figure 5, P2 explained that although the demonstration of the blackboard is 'good', the image does not depict the person of short stature 'realistically' as she resembles more 'a child wearing a uniform surrounded by average height students'.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.3 Evaluating AI · p. 14Scores were lowered if images misaligned with the prompt, for example by missing or mis-representing key aspects.
“Furthermore, scores were lowered if images misaligned with the prompt , for example by missing or mis-representing key aspects (e.g., no evidence of the person being a 'teacher').”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ 4.3 Evaluating AI · p. 14Project leads expressed their community-specific representation preferences across the full spectrum of the 5-point rating scale.
“Project leads expressed their community-specific representation preferences across the full spectrum of the 5-point rating scale, designed to capture holistic judgments across ∼ 300 AI images as input to building AI evaluator models.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.3 Evaluating AI · p. 14Variances between good or very good images were often attributed to how respectful the depiction felt, the mood captured, and the fit of smaller details with annotators' expectations.
“Variances between good or very good images were often attributed to how respectful the depiction felt; the mood captured (e.g., is the person showing enjoyment); and the fit of smaller details with annotators expectations (e.g., there should be more tools if it is to show a repair setting).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15Each community defined representation in their own distinct ways, reflected in diversity across their representation themes and within themes such as 'work' and 'diversity'.
“Each community defined representation in their own, distinct ways, which is reflected in diversity across their representation themes that spanned family, celebrations, or sports as well as within themes such as those of 'work' and 'diversity'.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 12This approach produced rich descriptions that foreground the representational relevance of an image to external audiences or AI models, and served as the basis for image-level prompt generation for AI evaluation.
“As exemplified in Figure 4, this approach produced rich descriptions that foreground the representational relevance of an image to external audiences, or AI models; as well as serving as the basis for image-level prompt generation for AI evaluation.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 12To include children, P1 leveraged existing contacts with schools for classroom captures and securing guardian approvals; whereas P3 bypassed guardian consent altogether by having adult members with dwarfism donate their childhood images.
“To include children, P1 leveraged existing contacts with schools for classroom captures and securing guardian approvals; whereas P3 bypassed guardian consent altogether by having adult members with dwarfism donate their childhood images.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 12P1 consulted members on consent protocols for images showing more than one individual; jointly they agreed to exclude images showing strangers, and to seek consent from known others.
“For example, P1 consulted members on consent protocols for images that showed more than one individual (e.g., should the person owning the image be able to give consent on others' behalf?); jointly they agreed to exclude images that showed strangers, and to seek consent from known others (e.g., family).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 4.2 Curating the Community Library · p. 12Within this configuration, project leads were able to specify preferred terminology for their community, reflecting their cultural values and ensuring accurate, respectful and inclusive language.
“Within this configuration, project leads were also able to specify preferred terminology for their community, reflecting their cultural values and ensuring accurate, respectful and inclusive language .”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.2 Intersecting Human Insights with Technical AI Requirements · p. 15The Community Library served as a useful reference for organizing project leads' thinking and outreach activities by reducing uncertainties about how much data and what type of data to collect for a target dataset, and its contents help preserve the richness and authenticity of community perspectives.
“While this process can be effortful, we found the Community Library to have served as a useful reference for organizing project leads thinking and outreach activities by reducing uncertainties about how much data and what type of data to collect for a target dataset (here: 400 images), whilst the contents themselves help preserve the richness and authenticity of community perspectives.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ A.3 Additional AI Evaluation Examples · p. 28The AI-generated image in Figure 15 was rated 'very good' by the project lead representing the vision impairment community (P1).
“The image in Figure 15 was rated as 'very good' by the project lead representing the vision impairment community (P1), who considered several factors: the realism of the image; how well it aligned with the prompt (e.g., if it depicted a computer class); the overall authenticity of the composition, and accuracy of the AT”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ A.3 Additional AI Evaluation Examples · p. 28The AI-generated image in Figure 16 was rated 'good' by the project lead of the Global South dwarfism community (P2).
“The AI-generated image in Figure 16 was rated as 'good' by the project lead of the Global South dwarfism community (P2).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ 5.3 Human-Centred AI Data Work · p. 16A deliberate focus on 'good' representation did not mean that negative examples would not surface; all project leads articulated common problems with stereotyping and lived experiences of disability used to anchor expressions of 'better' representation and deepening their definition.
“However, our findings showed how a deliberate focus on 'good' representation, did not mean that negative examples would not surface. All project leads articulated common problems with stereotyping and lived experiences of disability used to anchor expressions of 'better' representation and deepening their definition.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 5.3 Human-Centred AI Data Work · p. 16Their emphasis on defining 'good' representation was experienced positively by project leads, who described enjoyment collaborating with and learning from community members, expressed pride in the resulting image collection, and intentions to continue building on these efforts.
“Finally, our emphasis on defining 'good' representation was experienced positively by project leads, who described their enjoyment collaborating with and learning from their community members -especially during workshops and field trips. They expressed pride in the resulting image collection, and intentions to continue building on these efforts.”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning§ 5.3 Human-Centred AI Data Work · p. 16Such articulations of negative narratives were not imposed upon project leads, but evolved naturally, leaving them in control as to when and how they would engage with negative instances.
“Importantly, such articulations of negative narratives were not imposed upon project leads, but evolved naturally, leaving them in control as to when and how they would engage with negative instances.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ 5.3 Human-Centred AI Data Work · p. 17Each organization mobilized their community differently, and this shaped participatory inclusion and deliberation.
“Our findings show how each organization mobilized their community differently (Table 2), and how this shaped participatory inclusion and deliberation.”
Quote in the paperSubstantial claimMay combine two claims.
Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3Mack et al. found assistive technologies were frequently foregrounded over the individuals themselves in image generation, leading participants to describe such imagery as 'dehumanizing'.
“Additionally, assistive technologies (ATs) were frequently foregrounded over the individuals themselves, leading participants to describe such imagery as 'dehumanizing' [77].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [77] [77] Kelly Avery Mack, Rida Qadri, Remi Denton, Shaun K. Kane, and Cynthia L. Bennett. 2024. 'They only care to show us the wheelchair': disability representation in text-to-image AI models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 288, 23 pages. doi:10.1145/3613904.3642166 - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisereasoning§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3Addressing knowledge gaps through large-scale model pre-training is challenging due to the scale of data required, computational resource needs, and the proprietary nature of many advanced models.
“Addressing these knowledge gaps through large-scale model pre -training is challenging due to the scale of the data required, computational resource needs, and the proprietary nature of many advanced models.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3These framings typically reflect how disability is perceived by others rather than capturing the lived experiences of people with disabilities, resulting in inauthentic or incomplete depictions.
“These framings typically reflect how disability is perceived by others rather than capturing the lived experiences of PwD, resulting in inauthentic or incomplete depictions [49].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [49] [49] Tina Goethals, Dimitri Mortelmans, Hilde Van den Bulck, Willem Van den Heurck, and Geert Van Hove and. 2022. I am not your metaphor: frames and counter-frames in the representation of disability. Disability & Society 37, 5 (2022), 746-764. arXiv:https://doi.org/10.1080/09687599.2020.1836478 doi:10. 1080/09687599.2020.1836478 - source
Probably not a substantial claim: may not pass S4 (based).
Premisereasoning§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3A noted current challenge is the required foundational step of clearly defining what constitutes 'good' representation for a community that can be used in the fine-tuning process.
“A noted current challenge is the required foundational step of: clearly defining what constitutes 'good' representation for a community that can be used in the fine-tuning process.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3Positive portrayals of people with disabilities in visual media can affirm identity.
“It also impacts how PwD form their identity: positive portrayals can affirm identity, while negative ones may lead to denial [120].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [120] [120] L. Zhang and B. Haller. 2013. Consuming Image: How Mass Media Impact the Identity of People with Disabilities. Communication Quarterly 61, 3 (2013), 319-334. doi:10.1080/01463373.2013.776988 - source
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 2.1 Disability Representation in (Digital) Media · p. 3Negative portrayals of people with disabilities in visual media may lead to denial.
“It also impacts how PwD form their identity: positive portrayals can affirm identity, while negative ones may lead to denial [120].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [120] [120] L. Zhang and B. Haller. 2013. Consuming Image: How Mass Media Impact the Identity of People with Disabilities. Communication Quarterly 61, 3 (2013), 319-334. doi:10.1080/01463373.2013.776988 - source
Classified as probably not a substantial claim. May combine two claims.
Premisecitation§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4Data stewards - individuals or organizations - govern data on behalf of beneficiaries, involving community members and safeguarding community-specific materials by defining their use terms.
“Data stewards -individuals or organizations -govern data on behalf of beneficiaries [1, 64], involving community members and safeguarding community-specific materials by defining their use terms [25, 63] (e.g., via licensing).”
Quote in the paperSubstantial claim0 of 4 sources checked- cites [1] [1] British Academy and the Royal Society. 2017. Data management and use: governance in the 21st century. -
Not checked in the free run.
- cites [64] [64] Open Data Institute. 2020. Data Trusts in 2020. Available from https://theodi. org/news-and-events/blog/data-trusts-in-2020/.[Accessed17-04-2024]. -
Not checked in the free run.
- cites [25] [25] Madiha Zahrah Choksi, Sharon Heung, Reeda Shimaz Huda, Rita Faia Marques, Anja Thieme, and Cecily Morrison. [n. d.]. Terms of Care: Designing Participatory Data Governance for Disability Communities. ([n. d.]). [Manuscript in preparation]. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [63] [63] Ada Lovelace Institute. 2021. Participatory Data Stewardship: A Framework for Involving People in the Use of Data. Report (2021). -
Not checked in the free run.
May combine two claims.
Premisecitation§ 2.2 Community-Defined Concepts and Data for AI Evaluation & Image Generation · p. 3Common image generation evaluation metrics tend to over-index on superficial realism and lack the domain-specific knowledge needed to assess nuanced aspects of disability representation.
“They also tend to over-index on superficial realism and lack the domain-specific knowledge needed to assess nuanced aspects of disability representation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ 2.3 Bringing (Disability) Communities into AI Data Practices · p. 4Dataset curation involves more than 'technical' work.
“This underscores how dataset curation involves more than 'technical' work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ 4.1 Defining Representation · p. 10The definition of positive representation in contrast to negative examples highlights the importance of context, whether drawn from personal experiences or media history, in articulations of 'good' representation.
“This definition of positive representation in contrast to negative examples highlights the importance of context -whether drawn from personal experiences or media history -in articulations of 'good' representation .”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone given§ 3.1 Disability Communities & Project Leads · p. 4Visual characteristics of dwarfism are inseparable from the person.
“While both groups reflect 'visible disabilities' (vs. neurodiversity), they pose distinct challenges for AI: visual characteristics of dwarfism are inseparable from the person, whereas the identification of someone who is blind or has low vision often requires external identifiers (e.g., a guide cane).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone given§ 3.1 Disability Communities & Project Leads · p. 4Disability advocacy organizations have defined memberships and existing structures of communication and negotiation to support deliberations for defining visual identity.
“Such organizations have defined memberships and existing structures of communication and negotiation to support deliberations for defining visual identity.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone given§ 3.1 Disability Communities & Project Leads · p. 4Identification of someone who is blind or has low vision often requires external identifiers (e.g., a guide cane).
“While both groups reflect 'visible disabilities' (vs. neurodiversity), they pose distinct challenges for AI: visual characteristics of dwarfism are inseparable from the person, whereas the identification of someone who is blind or has low vision often requires external identifiers (e.g., a guide cane).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ Project Lead: P2 · p. 5Project leads are given a unique identification number to protect their anonymity.
“Project leads are given a unique identification number to protect their anonymity (P1-P3).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationreasoning§ 3.1 Disability Communities & Project Leads · p. 4Using disability advocacy organizations as proxies sets appropriate boundaries for determining the level of diversity needed in imagery to represent a community and avoids assuming a shared identity based solely on disability.
“This approach also sets appropriate boundaries for determining the level of diversity needed in imagery to represent a community and, most importantly, avoids assuming a shared identity based solely on disability.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone given§ Project Lead: P2 · p. 5The research study was carefully reviewed for compliance and IRB approved (Reference ID: 11019).
“The research study was carefully reviewed for compliance and IRB approved (Reference ID: 11019).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given§ Project Lead: P2 · p. 5Informed consent was sought in writing prior to the study.
“Informed consent was sought in writing prior to the study.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ 4.1 Defining Representation · p. 10Although organizations shared the importance of certain representation themes, each defined them in their own way.
“This highlights that although organizations shared the importance of certain representation themes, each defined them in their own way .”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ 4.1 Defining Representation · p. 10There is a distinction between representing a community through an 'advocacy organization' versus a group of 'people with disabilities', for which boundaries of inclusion can be difficult to define.
“This illustrates the distinction between representing a community through an 'advocacy organization' versus a group of 'people with disabilities' for which boundaries of inclusion however can be difficult to define .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone given§ 3.3 Research Data: Capture & Analysis · p. 8All research meetings were recorded and transcribed via Microsoft Teams, and key segments were checked for correctness before transcripts were anonymized and recordings deleted.
“All research meetings were recorded and transcribed via Microsoft Teams. Key segments were checked for correctness before transcripts were anonymized and recordings deleted.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationnone given§ 5 Discussion: Bringing Communities Meaingfully into AI Data Practices · p. 15To advance disability representation in image generation models requires a reconfiguration of how disability communities are meaningfully brought into AI data practices that underpin model development and evaluation.
“In this paper, we argue that to advance disability representation in image generation models requires a reconfiguration of how disability communities are meaningfully brought into AI data practices that underpin model development and evaluation.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisereasoning§ COMMUNITY-LED ACTIVITIES · p. 8A 'balanced' structure is necessary to prevent data skew or bias, where overrepresented themes or sub-themes could disproportionately influence the behavior of models trained on this data.
“This 'balanced' structure is necessary to prevent data skew or bias, where overrepresented themes or sub-themes could disproportionately influence the behavior of models trained on this data.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisecitation§ COMMUNITY-LED ACTIVITIES · p. 8Image annotations, including prompts and spatial cues, are essential for training and evaluating image generation models as they provide localized and semantically rich supervision that help models learn meaningful visual representations.
“Image annotations, including prompts and spatial cues [92, 121], are essential for training and evaluating image generation models as they provide localized and semantically rich supervision that help models learn meaningful visual representations.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [92] [92] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning . PmLR, 8748-8763. -
Not checked in the free run.
- cites [121] [121] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision . 3836-3847. - source
May combine two claims.
Interpretationown data§ 4.2 Curating the Community Library · p. 12Communities exercised agency in shaping consent strategies and leveraged their networks to meet their inclusion goals.
“These examples show how communities exercised agency in shaping consent strategies and leveraged their networks to meet their inclusion goals .”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationreasoning§ 4.2 Curating the Community Library · p. 12Themes are not equal, and this highlights limitations in this approach, where some themes may need more images or finer sub-theme distinctions.
“This underscores that themes are not equal and highlights limitations in this approach, where some themes may need more images or fi ner sub-theme distinctions .”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ 5.1 Technology-Supported, Community-Led Representation Definition · p. 15Defining representation is fundamentally a discursive process, shaped by negotiation and dialogue within communities, in response to broader societal narratives, and through material engagements with the images.
“Our work also demonstrates how defining representation is fundamentally a discursive process, shaped: (i) by negotiation and dialogue within communities (e.g., deliberations of urban vs. rural depictions); (ii) in response to broader societal narratives (e.g., desires to challenge existing stereotypes and common misconceptions); and (iii) through material engagements with the images.”
Quote in the paperSubstantial claimMay combine two claims.
Premisereasoning§ COMMUNITY-LED ACTIVITIES · p. 8A balanced structure enables controlled comparisons across themes and sub-themes, supporting disaggregated and more interpretable evaluation.
“It also enables controlled comparisons across themes and sub-themes, supporting disaggregated and more interpretable evaluation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ 3.3 Research Data: Capture & Analysis · p. 8Qualitative content analysis supports concurrent data collection and analysis, review of each study case alongside the data as a whole, and different content types.
“It supports: for data collection and analysis to occur concurrently; the review of each study case (e.g., individual community) alongside the data as a whole [37]; as well as different content types [70] -from research notes and transcripts to system entries and rating results.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [37] [37] Jane Forman and Laura Damschroder. 2007. Qualitative content analysis. In Empirical methods for bioethics: A primer . Emerald Group Publishing Limited, 39-62. - source
- cites [70] [70] Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology . Sage publications. - source
Probably not a substantial claim: may not pass S2 (checkable).
Premisecitation§ 3.3 Research Data: Capture & Analysis · p. 8Qualitative content analysis is well suited to the evolving process.
“This method is well suited to our evolving process [37, 70].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [37] [37] Jane Forman and Laura Damschroder. 2007. Qualitative content analysis. In Empirical methods for bioethics: A primer . Emerald Group Publishing Limited, 39-62. - source
- cites [70] [70] Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology . Sage publications. - source
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationown data§ 5.3 Human-Centred AI Data Work · p. 16Such experiences highlight the value of taking a human-centered approach to AI data practices that focuses on cultivating positive representation and prioritizes the meaning that communities derive from participating over the effort or time required.
“Such experiences highlight the value of taking a human-centered approach to AI data practices that focuses on cultivating positive representation; and prioritizes the meaning that communities derive from participating over the effort or time required -as might be considered in more transactional views.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ 5.3 Human-Centred AI Data Work · p. 17Taking a community-centric approach to AI data practices gives one perspective on 'who' gets to be involved, and 'whose values and norms' get to shape AI pipelines.
“Taking a community-centric approach to AI data practices gives us one perspective on 'who' gets to be involved, and 'whose values and norms' get to shape AI pipelines.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationnone given§ 6 Conclusion · p. 17Improving image generation requires more than just data.
“Recognizing that improving image generation requires more than just data, we placed communities and their advocacy organizations at the center of decisions about their own representation.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ 5.3 Human-Centred AI Data Work · p. 17While no form of participation or negotiated outcome will be perfectly inclusive, this approach ensures that decisions about whose voices are represented in the data remain with the communities themselves, rather than external AI developers.
“While no form of participation or negotiated outcome will be perfectly inclusive, this approach ensures that decisions about whose voices are represented in the data remain with the communities themselves, rather than external AI developers .”
Quote in the paperSubstantial claim- Rests on an assumption: No form of participation or negotiated outcome will be perfectly inclusive.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoning§ 5.4 Limitations & Future Work · p. 17A similar generalizability challenge applies to the future development of community-specific evaluator models, whose training data is constrained to the three image generation models used in the AI rating task.
“A similar generalizability challenge applies to the future development of community-specific evaluator models, whose training data is constraint to the three image generation models used in our AI rating task.”
Quote in the paperNot a substantial claim- Rests on an assumption: Different image generation models exhibit distinct limitations and failure modes.
May combine two claims.
Study design and conductreasoning§ 5.4 Limitations & Future Work · p. 17While the approach seeks to improve overall representation quality, it remains limited in its ability to eliminate model-specific errors.
“As a result, while our approach seeks to improve overall representation quality, it remains limited in its ability to eliminate model-specific errors.”
Quote in the paperNot a substantial claim- Rests on an assumption: Different image generation models exhibit distinct limitations and failure modes.
Classified as probably not a substantial claim.
Study design and conductnone given§ 5.4 Limitations & Future Work · p. 17This work is grounded in engagements with three disability organizations to enable in-depth, iterative dialogue about how project leads navigated their involvement in AI data practices and community engagement.
“This work has several limitations. It is grounded in engagements with three disability organizations to enable in-depth, iterative dialogue about how project leads navigated their involvement in AI data practices and community engagement.”
Quote in the paperSubstantial claim
The paper relies on this without support.
There remains limited understanding of what it means to represent people with disabilities and their communities well in (digital) media to drive meaningful change.
“Yet, there remains limited understanding of what it means to represent PwD and their communities well in (digital) media to drive meaningful change.”
The paper relies on this without support.
Data alone is not enough to improve AI models.
“Data alone is not enough to improve AI models.”
The paper relies on this without support.
Evaluation approaches are in their infancy.
“This growing body of work underscores that evaluation approaches are in their infancy, and that solutions must be both technically practical and socially situated.”
The paper relies on this without support.
Blind people in Kenya may have different views and visual identity compared to blind people in the United States.
“For instance, blind people in Kenya may have different views (e.g., what do I want the world to see) and visual identity (e.g., different ATs) compared to blind people in the United States.”
The paper relies on this without support.
Power dynamics between advocacy organizations and their members may vary.
“We acknowledge that power dynamics between advocacy organizations and their members may vary.”
The paper relies on this without support.
Community involvement in AI data practices, if present, has often been limited to data collection, annotation or model output evaluation.
“Community involvement, if present, has often been limited to data collection, annotation or model output evaluation.”
The paper relies on this without support.
Conceptualized representation and its translation in data are under-supported.
“Conceptualized representation and its translation in data are under-supported.”
The paper relies on this without support.
AI evaluation processes require well-defined concepts to evaluate for success.
“This step was motivated by AI evaluation processes requiring well-defined concepts to evaluate for success.”
The paper relies on this without support.
Project leads are advocates embedded within their community.
“As advocates embedded within their community, project leads self-organized their outreach using familiar methods for engaging with their members.”
The authors state this as an assumption.
Current AI image generation misrepresents disability communities.
“we believe this is a pivotal time to address misrepresentation in AI and are committed to centering communities in AI development”
The paper relies on this without support.
Community members can meaningfully contribute to defining representation for AI.
“Recognizing the influential role of project leads in defining community representation, it is essential to also involve community members directly into the definition process.”
The paper relies on this without support.
The latest image generation models are appropriate for grounding community representation definitions.
“Phase 3 grounds each community's representation definition in the outputs of the latest image generation models to better align the data with downstream model alignment and evaluation needs.”
The paper relies on this without support.
Different image generation models exhibit distinct limitations and failure modes.
“However, different image generation models exhibit distinct limitations and failure modes.”
The paper relies on this without support.
No form of participation or negotiated outcome will be perfectly inclusive.
“While no form of participation or negotiated outcome will be perfectly inclusive, this approach ensures that decisions about whose voices are represented in the data remain with the communities themselves, rather than external AI developers .”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Organization 2 outreach count does not match its breakdown
MinorThe listed activities account for 37 individual interviewees plus 9 members in each of two group interviews, totaling 55 people rather than the reported 57.
Reported 57 · recomputed 55
“Consultation of 57 people of short stature via: (a) Individual online in- terviews (37 members); (b) Two in- person group interviews (1.5 hours each, 9 members/ per session)”
Cites
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
One size fits all? What counts as quality practice in (reflexive) thematic analysis?
Braun, V., & Clarke, V. (2021) · Qualitative Research in Psychology · doi:10.1080/14780887.2020.1769238 · Cited by 9,727 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Sorting Things Out: Classification and Its Consequences
Bowker, G. C., & Star, S. L. (1999) · doi:10.7551/mitpress/6352.001.0001 · Cited by 6,188 · see citing works
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547 · see citing works
Colman (2022) · doi:10.1073/pnas · Cited by 4,370 · see citing works
A worked example of Braun and Clarke’s approach to reflexive thematic analysis
Byrne, D. (2022) · Quality & Quantity · doi:10.1007/s11135-021-01182-y · Cited by 3,604 · see citing works
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 2× · see citing works
“12: This screenshot shows the second frame (2/3) of the per-image data annotation sequence: project leads review and can make any necessary edits to the displayed prompt that is auto-generated using the image and their initial explanation for selecting it as inputs to the GPT4o model.”
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.11487 · Cited by 2,528 · see citing works
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1× · see citing works
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · see citing works
Braun, V., & Clarke, V. (2021) · Analysing Qualitative Data in Psychology · doi:10.4135/9781036232078.n9 · Cited by 1,504 · see citing works
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., & Choi, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.595 · Cited by 1,232 · cited 1× · see citing works
“However, common image generation evaluation metrics — such as CLIPScore [55], FID [56], and uses of multimodal large language models (MLLMs) ‘as-judge’ metrics [58, 71] often penalize authentic depictions of ATs, while re-warding incorrect or even offensive renderings (e.g., Braille displays shown…”
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · see citing works
Forman, J., & Damschroder, L. (2007) · Advances in Bioethics · doi:10.1016/s1479-3709(07)11003-7 · Cited by 808 · see citing works
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · cited 3× · see citing works
“…our approach to providing disability advocacy organizations with technology scaffolds and facilitating opportunities for community dialogue, which enabled the production of nuanced, contextual accounts of what ‘good’ representation means for each community, and its embedding within data for AI.”
Photorealistic Text-To-Image Diffusion Models with Deep Language Understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, K., Lopes, R. G., Ayan, B. K., Salimans, T., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2643 · Cited by 559 · see citing works
Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
Bianchi, F., Kalluri, P., Durmus, E., Ladhak, F., Cheng, M., Nozza, D., Hashimoto, T., Jurafsky, D., Zou, J., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594095 · Cited by 342 · cited 2× · see citing works
“Previous research has highlighted challenges for image generators to depict PwD in respectful and realistic ways [48, 77], echoing broader biases seen in representations of other marginalized communities [12, 21, 81, 91].”
LAION-5B: An Open Large-Scale Dataset for Training Next Generation Image-Text Models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1833 · Cited by 325 · see citing works
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275 · cited 2× · see citing works
“12: This screenshot shows the second frame (2/3) of the per-image data annotation sequence: project leads review and can make any necessary edits to the displayed prompt that is auto-generated using the image and their initial explanation for selecting it as inputs to the GPT4o model.”
Indigenous Data Governance: Strategies from United States Native Nations
Carroll, S. R., Rodriguez-Lonebear, D., & Martinez, A. (2019) · Data Science Journal · doi:10.5334/dsj-2019-031 · Cited by 264 · cited 1× · see citing works
“AI data stewardship applies this approach to dataset collection for training AI models, especially when these have knowledge gaps about marginalized communities (cf. [20, 59]).”
Large image datasets: A pyrrhic win for computer vision?
Birhane, A., & Prabhu, V. U. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00158 · Cited by 257 · cited 1× · see citing works
“Current data collection processes prioritize scale over inclusion, relying on web-scraping (e.g., [40, 97]) and synthetic pipelines that reflect developer biases and the uneven availability of online content [13, 14, 57, 96].”
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256 · cited 7× · see citing works
“…crucial for building trust and rapport with community members [53]; (2 ) the importance of information scaffolds and other educational resources to making participation accessible [31, 53, 107]; and (3) the need to contextualize community values to support meaningful data collection [11, 53, 90].”
InfluentialAya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Singh, S., Vargus, F., Dsouza, D., Karlsson, B. F., Mahendiran, A., Ko, W.-Y., Shandilya, H., Patel, J., Mataciunas, D., O'Mahony, L., et al. (2024) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2402.06619 · Cited by 249 · cited 1× · see citing works
“To address representational gaps and biases in AI models, a number of community-centric datasets are emerging aimed especially at improving cultural and geographical representation (e.g., DOSA [99], World Wide Dishes [53, 78], or GeoDE [93]); and language diversity (e.g., Masakhane [84], Aya [102]).”
Bottom-up data Trusts: disturbing the ‘one size fits all’ approach to data governance
Delacroix, S., & Lawrence, N. D. (2019) · International Data Privacy Law · doi:10.1093/idpl/ipz014 · Cited by 179 · see citing works
CLIPScore: A Reference-free Evaluation Metric for Image CaptioningPreprint
Hessel, J., Holtzman, A., Forbes, M., Bras, R. J. L., & Choi, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08718 · Cited by 174 · cited 1× · see citing works
“However, common image generation evaluation metrics — such as CLIPScore [55], FID [56], and uses of multimodal large language models (MLLMs) ‘as-judge’ metrics [58, 71] often penalize authentic depictions of ATs, while re-warding incorrect or even offensive renderings (e.g., Braille displays shown…”
"I wouldn't say offensive but...": Disability-Centered Perspectives on Large Language Models
Gadiraju, V., Kane, S., Dev, S., Taylor, A., Wang, D., Denton, R., & Brewer, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3593989 · Cited by 155 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
Consuming Image: How Mass Media Impact the Identity of People with Disabilities
Zhang, L., & Haller, B. (2013) · Communication Quarterly · doi:10.1080/01463373.2013.776988 · Cited by 152 · cited 2× · see citing works
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] – portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering
Hu, Y., Liu, B., Kasai, J., Wang, Y., Mari Ostendorf, E., Krishna, R., & Smith, N. A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01866 · Cited by 151 · cited 1× · see citing works
“…image generation evaluation metrics — such as CLIPScore [55], FID [56], and uses of multimodal large language models (MLLMs) ‘as-judge’ metrics [58, 71] often penalize authentic depictions of ATs, while re-warding incorrect or even offensive renderings (e.g., Braille displays shown as paper)…”
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.03206 · Cited by 140 · see citing works
Bennett, C. L., Gleason, C., Scheuerman, M. K., Bigham, J. P., Guo, A., & To, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445498 · Cited by 128 · cited 1× · see citing works
“Several works have explored how PwD wish to represent themselves in digital depictions, mainly as virtual avatars or agents (e.g., [9, 33, 43, 76, 118, 119]).”
Toward human-centered algorithm design
Baumer, E. P. (2017) · Big Data & Society · doi:10.1177/2053951717718854 · Cited by 125 · cited 1× · see citing works
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
Generative AI and the politics of visibility
Gillespie, T. (2024) · Big Data & Society · doi:10.1177/20539517241252131 · Cited by 122 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
DataComp: In search of the next generation of multimodal datasets
Gadre, S. Y., Ilharco, G., Fang, A., Hayase, J., Smyrnis, G., Nguyen, T., Marten, R., Wortsman, M., Ghosh, D., Zhang, J., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1179 · Cited by 120 · see citing works
Envisioning Communities: A Participatory Approach Towards AI for Social Good
Bondi, E., Xu, L., Acosta-Navas, D., & Killian, J. A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462612 · Cited by 115 · cited 1× · see citing works
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
Indigenous Peoples' Rights in Data: a contribution toward Indigenous Research Sovereignty
Hudson, M., Carroll, S. R., Anderson, J., Blackwater, D., Cordova-Marks, F. M., Cummins, J., David-Chavez, D., Fernandez, A., Garba, I., Hiraldo, D., et al. (2023) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2023.1173805 · Cited by 107 · see citing works
The Routledge Companion to Disability and Media
(2019) · doi:10.4324/9781315716008 · Cited by 85 · cited 4× · see citing works
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] – portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
InfluentialAI’s Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia
Qadri, R., Shelby, R., Bennett, C. L., & Denton, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594016 · Cited by 85 · cited 6× · see citing works
“Previous research has highlighted challenges for image generators to depict PwD in respectful and realistic ways [48, 77], echoing broader biases seen in representations of other marginalized communities [12, 21, 81, 91].”
InfluentialIdentifying and Improving Disability Bias in GPT-Based Resume Screening
Glazko, K., Mohammed, Y., Kosa, B., Potluri, V., & Mankoff, J. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658933 · Cited by 85 · see citing works
Zhang, K., Deldari, E., Lu, Z., Yao, Y., & Zhao, Y. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544829 · Cited by 82 · see citing works
An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility
Glazko, K. S., Yamagami, M., Desai, A., Mack, K. A., Potluri, V., Xu, X., & Mankoff, J. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3614548 · Cited by 80 · cited 3× · see citing works
“As a consequence, they currently depict PwD in unrealistic or dehumanizing ways [48, 77], and often omit or distort representations of assistive technologies [77].”
Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generation
Otani, M., Togashi, R., Sawai, Y., Ishigami, R., Nakashima, Y., Rahtu, E., Heikkilä, J., & Satoh, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01372 · Cited by 78 · see citing works
Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation
Goyal, N., Kivlichan, I. D., Rosen, R., & Vasserman, L. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555088 · Cited by 76 · cited 3× · see citing works
“In expressing ‘their’ perspective through data annotations, it important to recognize how the identity and views of human annotators become embedded in the data, influencing subsequent system behavior [51, 52, 90].”
"I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools
Adnin, R., & Das, M. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675631 · Cited by 73 · see citing works
Best Prompts for Text-to-Image Models and How to Find Them
Pavlichenko, N., & Ustalov, D. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3592000 · Cited by 71 · see citing works
GenAssist: Making Image Generation Accessible
Huh, M., Peng, Y.-H., & Pavel, A. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606735 · Cited by 67 · see citing works
“They only care to show us the wheelchair”: disability representation in text-to-image AI models
Mack, K. A., Qadri, R., Denton, R., Kane, S. K., & Bennett, C. L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642166 · Cited by 67 · cited 8× · see citing works
“Previous research has highlighted challenges for image generators to depict PwD in respectful and realistic ways [48, 77], echoing broader biases seen in representations of other marginalized communities [12, 21, 81, 91].”
InfluentialDisability in influencer marketing: a complex model of disability representation
Södergren, J., & Vallström, N. (2023) · Journal of Marketing Management · doi:10.1080/0267257x.2022.2144418 · Cited by 65 · cited 2× · see citing works
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] – portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
2023 ACM Conference on Fairness Accountability and Transparency
(2023) · doi:10.1145/3593013 · Cited by 61 · see citing works
Investigating Gender and Racial Biases in DALL-E Mini Images
Cheong, M., Abedin, E., Ferreira, M., Reimann, R., Chalson, S., Robinson, P., Byrne, J., Ruppanner, L., Alfano, M., & Klein, C. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3649883 · Cited by 57 · cited 2× · see citing works
“Previous research has highlighted challenges for image generators to depict PwD in respectful and realistic ways [48, 77], echoing broader biases seen in representations of other marginalized communities [12, 21, 81, 91].”
VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation
Ku, M., Jiang, D., Wei, C., Yue, X., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2312.14867 · Cited by 56
Towards Inclusive Avatars: Disability Representation in Avatar Platforms
Mack, K., Hsu, R. C. L., Monroy-Hernández, A., Smith, B. A., & Liu, F. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581481 · Cited by 54 · cited 3× · see citing works
“Several works have explored how PwD wish to represent themselves in digital depictions, mainly as virtual avatars or agents (e.g., [9, 33, 43, 76, 118, 119]).”
Masakhane -- Machine Translation For Africa
Orife, I., Kreutzer, J., Sibanda, B., Whitenack, D., Siminyu, K., Martinus, L., Ali, J. T., Abbott, J., Marivate, V., Kabongo, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2003.11529 · Cited by 51 · cited 1× · see citing works
“To address representational gaps and biases in AI models, a number of community-centric datasets are emerging aimed especially at improving cultural and geographical representation (e.g., DOSA [99], World Wide Dishes [53, 78], or GeoDE [93]); and language diversity (e.g., Masakhane [84], Aya [102]).”
Unintended machine learning biases as social barriers for persons with disabilitiess
Hutchinson, B., Prabhakaran, V., Denton, R., Webster, K., Zhong, Y., & Denuyl, S. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386305 · Cited by 49 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
STELA: a community-centred approach to norm elicitation for AI alignment
Bergman, S., Marchal, N., Mellor, J., Mohamed, S., Gabriel, I., & Isaac, W. (2024) · Scientific Reports · doi:10.1038/s41598-024-56648-4 · Cited by 48 · cited 6× · see citing works
“This has led to calls for more inclusive, community-centric AI evaluation approaches [11, 68] that leverage qualitative methods like focus groups [89] and online workshops [68] to capture nuanced, socially situated and dynamic aspects of concepts like cultural or disability representation; as well…”
InfluentialKreiss, E., Bennett, C. L., Hooshmand, S., Zelikman, E., Morris, M. R., & Potts, C. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2205.10646 · Cited by 47
Mim, N. J., Nandi, D., Khan, S. S., Dey, A., & Ahmed, S. I. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641951 · Cited by 42 · cited 3× · see citing works
“Previous research has highlighted challenges for image generators to depict PwD in respectful and realistic ways [48, 77], echoing broader biases seen in representations of other marginalized communities [12, 21, 81, 91].”
Measuring Representational Harms in Image Captioning
Wang, A., Barocas, S., Laird, K., & Wallach, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533099 · Cited by 40 · cited 1× · see citing works
“Researchers have previously problematized anonymous crowdsourced data annotation approaches [31, 53, 65] as being unspecific about who’s worldviews and beliefs become manifest in the data [11, 17, 51, 78, 90, 91].”
Misfitting With AI: How Blind People Verify and Contest AI Errors
Alharbi, R., Lor, P., Herskovitz, J., Schoenebeck, S., & Brewer, R. N. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675659 · Cited by 40 · cited 1× · see citing works
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition
Desai, A., Berger, L., Minakov, F., Milano, N., Singh, C., Pumphrey, K., Ladner, R., III, H. D., Lu, A. X., Caselli, N., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3360 · Cited by 39 · see citing works
I am not your metaphor: frames and counter-frames in the representation of disability
Goethals, T., Mortelmans, D., Van den Bulck, H., Van den Heurck, W., & Van Hove, G. (2022) · Disability & Society · doi:10.1080/09687599.2020.1836478 · Cited by 38 · cited 3× · see citing works
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] – portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition
Massiceti, D., Zintgraf, L., Bronskill, J., Theodorou, L., Harris, M. T., Cutrell, E., Morrison, C., Hofmann, K., & Stumpf, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01064 · Cited by 33 · cited 1× · see citing works
“In disability contexts, key datasets include ORBIT [80] – videos and images of objects recorded by people who are blind or low-vision; StammerTalk [74] and ASL Citizen [32] for speech and language impairments; and the grassroots Disabled And Here stock image collection celebrating disabled Black,…”
Self-citationTaxonomizing and Measuring Representational Harms: A Look at Image Tagging
Katzman, J., Wang, A., Scheuerman, M., Blodgett, S. L., Laird, K., Wallach, H., & Barocas, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26670 · Cited by 32 · see citing works
Three Tensions Between Personas and Complex Disability Identities
Edwards, E. J., Sum, C. M., & Branham, S. M. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382931 · Cited by 30 · cited 1× · see citing works
“Several works have explored how PwD wish to represent themselves in digital depictions, mainly as virtual avatars or agents (e.g., [9, 33, 43, 76, 118, 119]).”
Everyday Uncertainty: How Blind People Use GenAI Tools for Information Access
Tang, X., Abdolrahmani, A., Gergle, D., & Piper, A. M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713433 · Cited by 30 · cited 1× · see citing works
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
Shelton, S. S., & Waddell, T. F. (2021) · Journal of Current Issues & Research in Advertising · doi:10.1080/10641734.2020.1808125 · Cited by 28 · cited 1× · see citing works
“Yet, PwD are frequently absent or negatively stereotyped in media [34, 49, 103, 120] – portraying them as objects of pity, charity, or humor [49, 88], or as super-heroes overcoming insurmountable barriers [34, 100, 120].”
The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models
Birhane, A., Dehdashtian, S., Prabhu, V., & Boddeti, V. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658968 · Cited by 28 · cited 1× · see citing works
“Current data collection processes prioritize scale over inclusion, relying on web-scraping (e.g., [40, 97]) and synthetic pipelines that reflect developer biases and the uneven availability of online content [13, 14, 57, 96].”
Challenging Ableism: A Critical Turn Toward Disability Justice in HCI
Sum, C. M., Spektor, F., Alharbi, R., Baltaxe-Admony, L. B., Devine, E., Dixon, H. A., Duval, J., Eagle, T., Elavsky, F., Fernandes, K., et al. (2024) · XRDS: Crossroads, The ACM Magazine for Students · doi:10.1145/3665602 · Cited by 28 · cited 1× · see citing works
“There is also a growing trend in proxy-based participation, whereby individuals familiar with a stakeholder community, including UX/HCI practitioners, often ‘stand-in’ for others [27, 31, 106] rather than enabling communities to speak for themselves.”
Human-Centered AI: A New Synthesis
Shneiderman, B. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-85623-6_1 · Cited by 27 · cited 1× · see citing works
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
Foundation Models in Healthcare: Opportunities, Risks & Strategies Forward
Thieme, A., Nori, A., Ghassemi, M., Bommasani, R., Andersen, T. O., & Luger, E. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3583177 · Cited by 25 · cited 1× · see citing works
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
Self-citationLee, S., Kohga, M., Landau, S., O'Modhrain, S., & Subramonyam, H. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675600 · Cited by 25 · see citing works
Seo, J., Kamath, S. S., Zeidieh, A., Venkatesh, S., & McCurry, S. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675660 · Cited by 25 · cited 1× · see citing works
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
Zhang, K., Spencer, E. G. S., Manikandan, A., Li, A., Li, A., Yao, Y., & Zhao, Y. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714230 · Cited by 25 · see citing works
Kreiss, E., Bennett, C., Hooshmand, S., Zelikman, E., Ringel Morris, M., & Potts, C. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.309 · Cited by 24 · see citing works
Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation
Chien, J., & Danks, D. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658946 · Cited by 22 · cited 1× · see citing works
“…‘good’ representation differs from existing research that predominantly focuses on understanding, defining or evaluating representational ‘harms’ [24, 28, 38, 44, 67, 77, 91, 116] and often uses poor AI-generated images to elicit feedback from marginalized communities (e.g., [65, 77, 79]).”
Suggs, D. W., & Guthrie, J. L. (2017) · International Journal of Sport Communication · doi:10.1123/ijsc.2017-0030 · Cited by 21 · cited 2× · see citing works
“How people with disabilities (PwD) are presented in media plays a pivotal role in shaping public perceptions and social norms [34, 105], influencing livelihood through access to education and employment [47].”
Inspecting the Geographical Representativeness of Images from Text-to-Image Models
Basu, A., Babu, R. V., & Pruthi, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00474 · Cited by 21 · cited 1× · see citing works
“The most dominant approach to gathering human annotations is to ‘crowd-source’ [7, 53, 65] either anonymous people or community members as ‘data workers’, who are given simple, tokenis-tic tasks “that even inexperienced annotators can finish in an in-stance” [31, 85], with little or no context to…”
Vertoont, S., Goethals, T., Dhaenens, F., Schelfhout, P., Van Deynse, T., Vermeir, G., & Ysebaert, M. (2022) · Critical Studies in Media Communication · doi:10.1080/15295036.2021.1979239 · Cited by 20 · cited 1× · see citing works
“Yet, media portrayals of PwD have historically been absent, stereotyped, harmful or inaccurate rather than reflecting authentic, lived experiences [34, 49, 88, 114, 117].”
Examining risks of racial biases in NLP tools for child protective services
Field, A., Coston, A., Gandhi, N., Chouldechova, A., Putnam-Hornstein, E., Steier, D., & Tsvetkov, Y. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594094 · Cited by 18 · see citing works
From Fitting Participation to Forging Relationships: The Art of Participatory ML
Cooper, N., & Zafiroglu, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642775 · Cited by 17 · cited 2× · see citing works
“There is also a growing trend in proxy-based participation, whereby individuals familiar with a stakeholder community, including UX/HCI practitioners, often ‘stand-in’ for others [27, 31, 106] rather than enabling communities to speak for themselves.”
ArtInsight: Enabling AI-Powered Artwork Engagement for Mixed Visual-Ability Families
Chheda-Kothary, A., Kanchi, R., Sanders, C., Xiao, K., Sengupta, A., Kneitmix, M., Wobbrock, J. O., & Froehlich, J. E. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712082 · Cited by 16 · cited 1× · see citing works
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
Herold, B. K. A., Waller, J. M., & Kushalnagar, R. (2022) · Workshop on Speech and Language Processing for Assistive Technologies · doi:10.18653/v1/2022.slpat-1.8 · Cited by 14 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
Tan, M., Lee, H., Wang, D., & Subramonyam, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637358 · Cited by 14 · cited 4× · see citing works
“…our approach to providing disability advocacy organizations with technology scaffolds and facilitating opportunities for community dialogue, which enabled the production of nuanced, contextual accounts of what ‘good’ representation means for each community, and its embedding within data for AI.”
InfluentialGerling, K., Depoortere, A., Wauters, J., Spiel, K., Alexandrovsky, D., Danckaerts, M., Baeyens, D., & der Oord, S. V. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3685276 · Cited by 14 · cited 1× · see citing works
“Several works have explored how PwD wish to represent themselves in digital depictions, mainly as virtual avatars or agents (e.g., [9, 33, 43, 76, 118, 119]).”
Huffman, S., Chen, S., Mack, K. A., Su, H., Wang, Q., & Kushalnagar, R. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3719785 · Cited by 14 · cited 1× · see citing works
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
Magomere, J., Ishida, S., Afonja, T., Salama, A., Kochin, D., Foutse, Y., Hamzaoui, I., Sefala, R., Alaagib, A., Dalal, S., et al. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732019 · Cited by 13 · cited 2× · see citing works
“Greater awareness of model specific errors could guide communities to curate data that explicitly targets these shortcomings.”
The representation of disabled women and recent disabled women-led media
Wilde, A. (2022) · Disability & Society · doi:10.1080/09687599.2021.2015292 · Cited by 11 · cited 3× · see citing works
“Visual media plays a significant role in shaping public opinion and social norms, including societal perceptions of disability [10, 34, 105, 117].”
Towards Geographic Inclusion in the Evaluation of Text-to-Image Models
Hall, M., Bell, S. J., Ross, C., Williams, A., Drozdzal, M., & Soriano, A. R. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658927 · Cited by 11 · cited 7× · see citing works
“Our work foregrounds defining ‘good’ representation for AI and adopts a model agnostic approach to data curation and annotation.”
InfluentialWho's in and who's out? A case study of multimodal CLIP-filtering in DataComp
Hong, R., Agnew, W., Kohno, T., & Morgenstern, J. (2024) · Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3689904.3694702 · Cited by 11 · cited 2× · see citing works
“Even when PwD data is available, data from marginalized groups is often automatically filtered out by techniques for removing ‘low’ quality data [57, 104], which rely on biased models such as CLIP [92].”
Bottom-up data Trusts: disturbing the ‘one size fits all’ approach to data governance
Delacroix, S., & Lawrence, N. (2018) · doi:10.2139/ssrn.3265315 · Cited by 11 · see citing works
DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures
Seth, A., Ahuja, S., Bali, K., & Sitaram, S. (2024) · doi:10.2139/ssrn.4756716 · Cited by 8 · see citing works
STILE: Exploring and Debugging Social Biases in Pre-trained Text Representations
Kabir, S., Li, L., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642111 · Cited by 8 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
Imagining the Far East: Exploring Perceived Biases in AI-Generated Images of East Asian Women
Lan, X., An, J., Guo, Y., Chiyou, T., Cai, X., & Zhang, J. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3719678 · Cited by 8 · cited 1× · see citing works
“Generative AI outputs tend to reproduce normative views and mirror societal biases for marginalized groups [15, 21, 36, 41, 42, 45, 53, 66, 72, 91], including PwD [10, 12, 39, 48, 54, 62, 77, 113].”
GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition
Ramaswamy, V., Lin, S. Y., Zhao, D., et al. (2023) · neural information processing systems · doi:10.52202/075280-2888 · Cited by 8 · cited 1× · see citing works
“To address representational gaps and biases in AI models, a number of community-centric datasets are emerging aimed especially at improving cultural and geographical representation (e.g., DOSA [99], World Wide Dishes [53, 78], or GeoDE [93]); and language diversity (e.g., Masakhane [84], Aya [102]).”
From Melting Pots to Misrepresentations: Exploring Harms in Generative AI
Gautam, S., Venkit, P. N., & Ghosh, S. (2024) · arXiv · doi:10.48550/arxiv.2403.10776 · Cited by 7 · see citing works
Ghosh, S., Lutz, N., & Caliskan, A. (2024) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v7i1.31650 · Cited by 7 · see citing works
Taxonomizing Representational Harms using Speech Act Theory
Corvi, E., Washington, H., Reed, S., Atalla, C., Chouldechova, A., Dow, P. A., Garcia-Gathright, J., Pangakis, N. J., Sheng, E., Vann, D., et al. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.202 · Cited by 6 · see citing works
The Case for "Thick Evaluations" of Cultural Representation in AI
Qadri, R., Diaz, M., Wang, D., & Madaio, M. (2025) · arXiv · doi:10.48550/arxiv.2503.19075 · Cited by 6
AI red-teaming is a sociotechnical problem: on values, labor, and harms
Gillespie, T., Shaw, R., Gray, M. L., & Suh, J. (2024) · arXiv · doi:10.48550/arxiv.2412.09751 · Cited by 6
Li, Q., & Wu, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687014 · Cited by 4 · cited 1× · see citing works
“In disability contexts, key datasets include ORBIT [80] – videos and images of objects recorded by people who are blind or low-vision; StammerTalk [74] and ASL Citizen [32] for speech and language impairments; and the grassroots Disabled And Here stock image collection celebrating disabled Black,…”
Tu, R.-C., Ma, Z.-A., Lan, T., Zhao, Y., Huang, H., & Mao, X.-L. (2024) · arXiv · doi:10.48550/arxiv.2411.15488 · Cited by 4
Bennett, C. L., Kane, S. K., & Harrington, C. N. (2025) · Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3757887.3763012 · Cited by 3 · cited 6× · see citing works
“Our findings show how reflection scaffolds and data structures provided through the Community Library Creator – such as Pin-board, Magazine, the Community Library, image annotations, and AI image ratings – served as a productive bridge for project leads to articulate their communities’…”
InfluentialConfusing the Map for the Territory
Qadri, R., Madaio, M., & Gray, M. L. (2025) · Communications of the ACM · doi:10.1145/3731655 · Cited by 2 · cited 8× · see citing works
“In line with arguments by [68, 89, 90], such approaches should recognize the inherently subjective meanings and perspectives that shape social concepts like representation, thereby moving beyond singular or objective standards towards embracing the richness of community-specific interpretations .”
InfluentialPosition: Evaluating Generative AI Systems Is a Social Science Measurement Challenge
Wallach, H., Desai, M., Cooper, A. F., Wang, A., Atalla, C., Barocas, S., Blodgett, S. L., Chouldechova, A., Corvi, E., Dow, P. A., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.00561 · Cited by 1 · cited 1× · see citing works
“Recent calls – in generative AI evaluation frameworks [115] – to separate the systematization of the concept to be measured, and the operationalization of the measure technically,further emphasizes the need to articulate what is evaluated before determining how to evaluate it.”
Explaining CLIP's performance disparities on data from blind/low vision users
Massiceti, D., Longden, C., Słowik, A., Wills, S., Grayson, M., & Morrison, C. (2023) · arXiv · doi:10.48550/arxiv.2311.17315 · Cited by 1 · see citing works
Self-citationDOSA: A Dataset of Social Artifacts from Different Indian Geographical SubculturesPreprint
Seth, A., Ahuja, S., Bali, K., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.14651 · Cited by 1 · see citing works
Provocation: Who benefits from "inclusion" in Generative AI?
Dalal, S., Hall, S. M., & Johnson, N. (2024) · arXiv · doi:10.48550/arxiv.2411.09102 · Cited by 1 · see citing works
Taxonomizing Representational Harms using Speech Act TheoryPreprint
Corvi, E., Washington, H., Reed, S., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.00928 · Cited by 1 · see citing works
The Human Labour of Data Work: Capturing Cultural Diversity through World Wide Dishes
Hall, S. M., Dalal, S., Sefala, R., Yuehgoh, F., Alaagib, A., Hamzaoui, I., Ishida, S., Magomere, J., Crais, L., Salama, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.05961
Kiden, S., Peter, O., Reyes-Cruz, G., Klyshbekova, M., Choi, S., Bergin, A. G., Waheed, M., Eke, D., Azim, T., Ramchurn, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.27361
T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation
Ma, Z.-A., Lan, T., Tu, R.-C., Liu, S.-H., Huang, H., Wu, Z., Xu, C., & Mao, X.-L. (2025) · arXiv · doi:10.48550/arxiv.2505.17897
Representations of Dwarfism: How to Laugh with Us, Not at Us
Pritchard, E. (2024) · The Cultural Legacy of Disney · doi:10.5040/9798216431176.ch-005
Disability in influencer marketing: a complex model of disability representation
Södergren, J., & Vallström, N. (2024) · Influencer Marketing · doi:10.4324/9781003559412-5
Suggs, D. W., & Guthrie, J. L. (2017) · doi:10.4135/9781526445384
Flagged first, then by citations
Research works printed but not linked: 8
- Anonymous. 2022. Seen on screen: The importance of disability representation. Available from https://www.nielsen.com/ [Accessed 30-08-2025].
- Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. 2024. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning.
- Madiha Zahrah Choksi Sharon Heung Reeda Shimaz Huda Rita Faia Marques Anja Thieme and Cecily Morrison. [n. d.]. Terms of Care: Designing Participatory Data Governance for Disability Communities. ([n. d.]). [Manuscript in preparation].
- 10.1145/3593013.3593989 · DOI · Paper page
- Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In International conference on machine learning. Pmlr, 8821–8831.
- Hariharan Subramonyam, Colleen Seifert, and MI Eytan Adar. 2021. How can human-centered design shape data-centric AI. In Proceedings of NeurIPS Data-Centric AI Workshop. Curran Associates New York, NY, USA, 3.
- Mei Tan Hansol Lee Dakuo Wang and Hari Subramonyam. 2024. Is a Seat at the Table Enough? Engaging Teachers and Students in Dataset Specification for ML in Education. Proc. ACM Hum.-Comput. Interact. 8 CSCW1 Article 81 (April 2024) 32 pages. 10.1145/3637358 · DOI · Paper page
- Pranav Narayanan Venkit, Mukund Srinath, and Shomir Wilson. 2022. A Study of Implicit Bias in Pretrained Language Models against People with Disabilities. In Proceedings of the 29th International Conference on Computational Linguistics, Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung-Hoon Na (Eds.). International Committee on Computational Linguistics, Gyeongju, Republic of Korea, 1324–1332. https://aclanthology.org/2022.coling-1.113/
Cited by
Sorted by citations
(2000) · Time to knit · doi:10.1145/3772318.3808924
- Authors
- Thieme, A., Faia Marques, R., Grayson, M., Balachandar, S., Tyler Cassidy, C., Choksi, M. Z., Longden, C., Huda, R. S., Ileve Kalovwe, N., Mallon, C., Mansperger, C., Massiceti, D., Mitra, B., Nzioka, R. M., Tanase, I., You, Y., & Morrison, C.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Microsoft Research, Cambridge, United Kingdom
- Microsoft Research, Cambridge, United Kingdom and Informatics, University of California, Irvine, California, USA
- Cornell Tech (US)
- Georgia Institute of Technology (US)
- Kilimanjaro Blind Trust Africa, Nairobi, Kenya
- Microsoft (United States) (US)
- LPA, Chicago, Illinois, USA
- Microsoft Research Montréal (Canada)
- Short Stature Society of Kenya, Nairobi, Kenya
- Microsoft Corporation, Toulouse, France
- University of Waterloo (CA)
- ORCID
- Venue
- Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2026-04-13
- Pages
- 1-29
- OpenAlex
- W7154168514
- Semantic Scholar
- 5f15a38604cebfe37644e4e896e16bae5a44de70
- Cited by
- 1
- References
- 84
- FWCI
- 0.00
- Type
- conference paper
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nc-nd)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Safety Research › Disability Rights and Representation
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- disability representation, AI image generation, community participation, disability community, human-centered AI, intersectionality, disability stereotypes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-25.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-25.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
