Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation

“This poses another challenge as even if suitable measures exist, they must first be appropriately calibrated (and may need to be regularly re-calibrated based on cultural and temporal shifts, see e.g. [56, 75, 83, 105]).”

“Allocative harms are relatively more easily measured, as they include physical, financial, social, and other observable resources (e.g., wages and jobs lost, loans not approved, houses not bought, or pharmaceutical interventions not pursued).”

“Table 2 provides just a few types of representational harm (with examples), as well as psychometrically validated measures/scales for that type of harm.”

“Through their lived experiences, these stakeholders can thereby ensure that the measurement is culturally and logistically appropriate and sustainable (see e.g. Participatory Research [60, 95]).”

“Group Environment Questionnaire [20], Perceived Cohesion Scale [16] Normalization or decreased likelihood of seeking psychological services among refugees or others who have experienced trauma [37, 47].”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“Allocative harms are relatively more easily measured, as they include physical, financial, social, and other observable resources (e.g., wages and jobs lost, loans not approved, houses not bought, or pharmaceutical interventions not pursued).”

“These definitions primarily focus on observing how one person acts toward another person or group, neglecting changes in internal cognitive states or reasoning, considerations of alternative possibilities, experiential fluctuations, self-identification, and similar psychological and social impacts.”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“…48, 65, 111, 115] Image Search, Image Tagging Discrimination/Incite (and Normalize) Hate/Violence [7, 48, 115, 128] LLMs Outsider Gaze [13, 23, 87] Generative AI, Story Generation Hierarchies & Marginalization [23, 111, 115] Image Tagging, Story Generation implementation, supported by a case study.”

“Thus, we need to adopt a more comprehensive characterization that is based in changes in mental representations, rather than the impacts of those changes.”

“Social Stereotyping [7, 13, 23, 31, 42, 48, 57, 65, 87, 101, 111, 115, 119, 123, 128, 134, 135] Image Captioning, Generative AI, ML, Image Cropping, Story Generation, Image Tagging, Knowledge Distillation Reification of Social Groups or Use of Es-sentialist Categories [7, 65, 111, 119, 123, 128,…”

“Allocative harms are relatively more easily measured, as they include physical, financial, social, and other observable resources (e.g., wages and jobs lost, loans not approved, houses not bought, or pharmaceutical interventions not pursued).”

“…workload, beliefs, desires, intentions, knowledges, thoughts, judgements, beliefs), affective (positive/negative valence, action-promoting/not), and emotional dimensions of mental representations that potentially result in harm (physical, psychological, social, or other) to an individual.”

“…look towards baseline estimation in media studies (e.g. measuring the impact of encountering specific representations from a film [8, 10, 43, 54, 64, 77, 84, 114]) and online education (e.g. measuring the impact of specific modules or training aimed at a student’s sense of belonging or…”

“Requesting users to complete surveys, particularly after every interaction, could introduce numerous biases (e.g., response bias, non-response bias, survivorship sampling bias) and decrease satisfaction [106].”

“Potential measures for these harms could include the Perceived Competence Scales (PCS) (a 4-item questionnaire) [131], the Feelings of Inadequacy scale (a 23-item self-reported measure) [14], and the Five-Factor Model of personality structure (consisting of 50 questions) [17, 130].”

“In the following section, we consider two aspects of elements of LLM’s seamless/frictionless design that pose the potential to exacerbate representational harms.”

“On this characterization, representational harms can arise across multiple temporal and social granularities (individual, intergroup, and intragroup).”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“These definitions primarily focus on observing how one person acts toward another person or group, neglecting changes in internal cognitive states or reasoning, considerations of alternative possibilities, experiential fluctuations, self-identification, and similar psychological and social impacts.”

“…Image Cropping, Story Generation, Image Tagging, Knowledge Distillation Reification of Social Groups or Use of Es-sentialist Categories [7, 65, 111, 119, 123, 128, 135] Image Captioning, Image Cropping, Image Tag-ging Inaccurate/Skewed Representations [44, 82, 119, 128, 135, 137] Misinformation,…”

“In addition, if the user had not continued on to ask about who is and isn’t represented in the responses, the harms after the first interaction (associating soccer stars as male) might be increased.”

“However, this model is not immune to mistakes, hallucinations, or irrelevant responses.”

“For instance, most LLM outputs are not accompanied with direct attribution, references to external/third-party sources or evidence in the training data.”

“This poses another challenge as even if suitable measures exist, they must first be appropriately calibrated (and may need to be regularly re-calibrated based on cultural and temporal shifts, see e.g. [56, 75, 83, 105]).”

“In the following section, we consider two aspects of elements of LLM’s seamless/frictionless design that pose the potential to exacerbate representational harms.”

“On this characterization, representational harms can arise across multiple temporal and social granularities (individual, intergroup, and intragroup).”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“This poses another challenge as even if suitable measures exist, they must first be appropriately calibrated (and may need to be regularly re-calibrated based on cultural and temporal shifts, see e.g. [56, 75, 83, 105]).”

“Thus, we need to adopt a more comprehensive characterization that is based in changes in mental representations, rather than the impacts of those changes.”

“…towards baseline estimation in media studies (e.g. measuring the impact of encountering specific representations from a film [8, 10, 43, 54, 64, 77, 84, 114]) and online education (e.g. measuring the impact of specific modules or training aimed at a student’s sense of belonging or inclusion…”

“Social Stereotyping [7, 13, 23, 31, 42, 48, 57, 65, 87, 101, 111, 115, 119, 123, 128, 134, 135] Image Captioning, Generative AI, ML, Image Cropping, Story Generation, Image Tagging, Knowledge Distillation Reification of Social Groups or Use of Es-sentialist Categories [7, 65, 111, 119, 123, 128,…”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“In the following section, we consider two aspects of elements of LLM’s seamless/frictionless design that pose the potential to exacerbate representational harms.”

“[11] Rather than manufacturing diversity (e.g., by flipping male and female names and roles, such as a victim for a robber), counter-narratives provide depth, complexity, and liberation through diversity of representation.”

“Table 2 provides just a few types of representational harm (with examples), as well as psychometrically validated measures/scales for that type of harm.”

“Counter-narratives have even been demonstrated to cause career benefits for minorities in STEM during high-school education [96].”

“This has the potential to make users themselves more susceptible to normative processes (i.e. epistemic harms), silently entrenching the dominant or loudest opinions that are most readily available, rather than tailoring to a user’s unique needs.”

“For example, a system that conveys multiple candidate responses for any given output may enable a user to get a better understanding of alternative perspectives and discourse within an issue.”

“Thus, we need to adopt a more comprehensive characterization that is based in changes in mental representations, rather than the impacts of those changes.”

“In the following section, we consider two aspects of elements of LLM’s seamless/frictionless design that pose the potential to exacerbate representational harms.”

“In order to understand the current landscape of representational harm definitions, we conducted a brief survey via snowball sampling using the keyword “representational harms” in the ACM Digital Library and in Google Scholar.”

“We thus propose an expansion of the definition of representational harms to include changes in cognitive (attention, engagement, workload, beliefs, desires, intentions, knowledges, thoughts, judgements, beliefs), affective (positive/negative valence, action-promoting/not), and emotional dimensions…”

“Thus, we need to adopt a more comprehensive characterization that is based in changes in mental representations, rather than the impacts of those changes.”

“In the following section, we consider two aspects of elements of LLM’s seamless/frictionless design that pose the potential to exacerbate representational harms.”

“In the following section, we consider two aspects of elements of LLM’s seamless/frictionless design that pose the potential to exacerbate representational harms.”

“Over-trust can also obscure the real, ongoing harms that AI systems already impose, including the ampli fi cation of social inequalities 65,66 , surveillance and erosion of privacy 67 , and the marginalization of vulnerable groups 68 .”

“Representational bias occurs when model outputs reflect or reinforce stereotypes about certain groups [21, 30, 41].”

“It demands active engagement and concrete actions from us as humans to address these issues, both in our AI systems and in society at large [38].”

“This can lead to significant representational harms, including behavioral, psychological, and cognitive impacts on individual and collective reputation, societal standing, and cultural cohesion [26].”

“Other work has used qualitative methods to characterize harms from LLMs that are more subtle and nuanced, and harder to automatically detect in a quantitative benchmark 99 – 103 .”

“This not only introduces further epistemic harms but also causes representa - tional harms (Chien & Danks, 2024 ) by perpetuating content that, despite pertaining to minority concerns, distorts or discounts minority perspectives.”

“Crawford (2017) used the term representational harms to refer to damages of this kind, which can lead to behavioral (e.g., discrimination, violence) as well as cognitive (e.g., intentions, beliefs), affective (e.g., sentiments), and emotional consequences (Chien & Danks, 2024).”

“Historical case studies, in particular, can help communicate why certain limitations of an AI system pose greater social risks than others, and thus aid the prioritization and orientation of mitigation strategies [27, 33, 160].”

“…‘good’ representation differs from existing research that predominantly focuses on understanding, defining or evaluating representational ‘harms’ [24, 28, 38, 44, 67, 77, 91, 116] and often uses poor AI-generated images to elicit feedback from marginalized communities (e.g., [65, 77, 79]).”

“…evidence, their quality rests on two conditions: the set must include the demographic and class locations that exist in the intended user population and the portrayals of those locations must preserve variation rather than rely on fixed scripts (Chien & Danks, 2024; Miaskiewicz & Kozar, 2011).”

“Outputs that appear neutral often encode majority or developer-preferred biases, entrenching representational harms (Chien & Danks, 2024) and heightening safety risks such as susceptibility to propaganda or cultural domination.”

“In this sense, the harms identified by job seekers extend beyond allocative or representational concerns toward psychological harms that affect users’ confidence, self-presentation, and emotional regulation through iterative engagement with the system [10, 12, 67].”

“The coders judged this to be a form of representational harm that could lead to stereotype threat (Section 2.2), which unnecessarily evokes feelings of exclusion or discriminatory backgrounds in relation to the reasons for selection [28].”