Fairness in representation: quantifying stereotyping as a representational harm

“This definition is based on the rep-resentativeness heuristic due to Kahnemann and Tver-sky [22, 23], a similarity heuristic that people rely on to judge the likelihood of uncertain events, instead of following the principles of probability theory [36].”

“This definition is based on the rep-resentativeness heuristic due to Kahnemann and Tver-sky [22, 23], a similarity heuristic that people rely on to judge the likelihood of uncertain events, instead of following the principles of probability theory [36].”

“There have been a few works that look at bias that emerges from the representation process, most notably when looking at learned representations that come from word-vector embeddings[4, 8, 33, 14].”

“The literature on algorithmic fairness has for the most part focused on bias caused by skew in training data distributions and the training process itself, and has quantified the bias in terms of fairness measures that are typically outcome-based (see, e.g., [7, 19, 39, 24] and surveys [29, 37]).”

“Some of the research in this area has taken a representational, preprocessing approach to reversing training data skew [17, 26, 40, 6, 16] by changing the inputs so that a classifier finds fair outcomes.”

“There have been a few works that look at bias that emerges from the representation process, most notably when looking at learned representations that come from word-vector embeddings[4, 8, 33, 14].”

“represent stereotyping. In the prototype model, people store and use an abstraction of stereotyped group’s typical features and judge the the said group members by their similarity to this prototype [9]. In the exemplar model, people use specic, real world individuals as representatives for the groups. As a group migh…”

“alized belief about a particular group or class of people [10]. Stereotypes can be positive: Asians are good at math, or negative: African-American names are more associated with criminal backgrounds [35]; Stereotypes are not limited to just racial groups and they can change over time [25]. One of the central purposes…”

“The literature on algorithmic fairness has for the most part focused on bias caused by skew in training data distributions and the training process itself, and has quantified the bias in terms of fairness measures that are typically outcome-based (see, e.g., [7, 19, 39, 24] and surveys [29, 37]).”

“As a group might have a number of exemplars; which one comes to mind when the stereotyping process is being activated depends on the context and situation in which the encounter with the stereotyped group member has occurred [32].”

“The literature on algorithmic fairness has for the most part focused on bias caused by skew in training data distributions and the training process itself, and has quantified the bias in terms of fairness measures that are typically outcome-based (see, e.g., [7, 19, 39, 24] and surveys [29, 37]).”

“In the context of unsupervised learning, recent work on fair clustering[12, 31, 3] and PCA[30, 28] seeks to generate a modified representation of the input points so that the new representation (clustering or reduced dimension) satisfies a notion of “balance” with respect to groups.”

“Some of the research in this area has taken a representational, preprocessing approach to reversing training data skew [17, 26, 40, 6, 16] by changing the inputs so that a classifier finds fair outcomes.”

“Some of the research in this area has taken a representational, preprocessing approach to reversing training data skew [17, 26, 40, 6, 16] by changing the inputs so that a classifier finds fair outcomes.”

“There have been a few works that look at bias that emerges from the representation process, most notably when looking at learned representations that come from word-vector embeddings[4, 8, 33, 14].”

“The literature on algorithmic fairness has for the most part focused on bias caused by skew in training data distributions and the training process itself, and has quantified the bias in terms of fairness measures that are typically outcome-based (see, e.g., [7, 19, 39, 24] and surveys [29, 37]).”

“The literature on algorithmic fairness has for the most part focused on bias caused by skew in training data distributions and the training process itself, and has quantified the bias in terms of fairness measures that are typically outcome-based (see, e.g., [7, 19, 39, 24] and surveys [29, 37]).”

“In the context of unsupervised learning, recent work on fair clustering[12, 31, 3] and PCA[30, 28] seeks to generate a modified representation of the input points so that the new representation (clustering or reduced dimension) satisfies a notion of “balance” with respect to groups.”

“omptly, they make us overlook individual Copyright c 2019 by SIAM Unauthorized reproduction of this article is prohibited arXiv:1901.09565v1 [cs.LG] 28 Jan 2019 dierences; this can lead to prejudice [34]. The approach we take in studying stereotypes is inspired by the literature on social cognition. This approach dene…”

“In the context of unsupervised learning, recent work on fair clustering[12, 31, 3] and PCA[30, 28] seeks to generate a modified representation of the input points so that the new representation (clustering or reduced dimension) satisfies a notion of “balance” with respect to groups.”

“In the context of unsupervised learning, recent work on fair clustering[12, 31, 3] and PCA[30, 28] seeks to generate a modified representation of the input points so that the new representation (clustering or reduced dimension) satisfies a notion of “balance” with respect to groups.”

“, LWPR [8], standard GPR [1], sparse online Gaussian process regression (OGP) [5] and υ-support vector regression (υ-SVR) [11], respectively.”

“For instance, stakeholders, including developers and implementers, will need to be mindful of allocational and representational harms 17,18 , particularly when a virtual agent acts as a gatekeeper 19 — in which case the patient-user has no other option than to interact fi rst with the virtual agent.”

“is possible to root the analysis in proxy groups [63], there is a risk of incorporating harmful stereotypes in proxy group definitions, potentially resulting in harms of representation [1].”

“A large body of works in the area of algorithmic fairness have focused on ensuring fair representation of all social groups in the machine learning pipeline (Bolukbasi et al., 2016; Samadi et al., 2018; Abbasi et al., 2019).”

“Hence, if unchecked, such representational harms in model predictions would percolate into allocational harms (cf. Crawford, 2017; Abbasi et al., 2019; Blodgett et al., 2020).”

“This may take the form of search results returning stereotypical images of minorities (Noble, 2018) or other algorithmic stereotyping (Abbasi et al., 2019); much of the work in word embeddings falls into this category (Caliskan et al., 2017), though research on downstream tasks and outcomes often…”

“Such misrepresentations can render certain people or groups less visible and promote stereotypes, with unpredictable but real effects (Abbasi et al., 2019).”

“representational, likely to take place when individuals and groups are unable to self-determine their image, which may end up being stereotyped, inadequate or offensive [1, 58, 62].”

“2We discuss this particular paper as it uncovers the hidden assumptions that seem to be held when algorithmic fairness scholars advocate for independence [1].”

“System users also experience representational harms via the reinforcement of stereotypes associating occupations with a particular gender, as well as lower quality of service in receiving grammatically incorrect translations (Abbasi et al., 2019).”

“Abbasi et al. (2019) discuss how harms from stereotypical representations manifest as allocative harms later in the ML pipeline.”

““Harms of representation” [1, 52] refers to how model predictions can reinforce potentially harmful stereotypes when trained on data exhibiting bias.”

“In an effort to prevent such transition of representational harms into allocational harms (cf. Crawford, 2017; Abbasi et al., 2019; Blodgett et al., 2020), we look at mitigating stereotyping biases at the source, i.”

“…— work especially poorly for some groups of workers (e.g., those with disabilities [75, 98]), leading to representation and allocation harms[1, 3, 132] When these technologies are designed to provide management with information about workers but fail to provide workers with the means to…”

“Our analysis investigates whether and how these demographic differences manifest across different demographic descriptors and prompt placement conditions.”

“For all (t, s) ∈ [1, 50], we perform s unique training runs (while changing both forms of randomness, i.”

“In addition to the representational harm [1] brought forward by”

“Historical discrimination has a huge impact on an individual’s starting point [62, 63, 64], and ongoing discrimination has a major impact on one’s income even for the same job (via gender [65] or ability [66]), income shocks [67], health shocks [68], and wealth shocks, as well as the likelihood of benefiting from publ…”

“We build on wider debates around the differences between what have been variously described as distributional versus dignitary [45] or allocative versus representational harms [1].”

“As emphasized by Jacobs and Wallach [60], operationalizing those constructs by measuring proxies may harm marginalized individuals and perpetuate stereotypes [1, 28, 107].”

“While stereotype can help an intuitive assessment of groups [11], it can easily lead to overgeneralization of individuals’ characteristics [1] based on their group membership.”

“The main algorithmic tools we use are drawn from the literature on fair clustering [3, 5, 10–12, 28, 29, 32, 36, 38, 39, 45], a sub-field of algorithmic fairness [2, 16, 19, 23, 30, 47] which has gained a lot of attention in recent years.”

“Such a demographic or social group is called protected group.”

“…stereotypical associations and implicit biases in large image-text multimodal models, particularly with regard to ‘measurable’ and therefore enactable categories of race, skin tone, and gender (Abbasi 2019; Birhane et al. 2021, 2024; Chen et al. 2023; Miltenburg 2016; Buolamwini and Gebru 2018).”

“Abbasi et al. [1] mathematically formalize stereotyping and show how models trained on stereotyped data can lead to un-fair outcomes.”

“Group fairness is a measure of performance disparity between different protected groups, rooted in concerns of algorithmic bias propagated from the dataset to the model [1,4,8,44].”

“Abbasi et al. [1] recently introduced a geometric notion of stereotyping.”

“Others examine if AI models and training data avoid stereotypical associations between demographic factors and educational outcomes (Abbasi et al., 2019 ; Binns, 2018 ).”

“FER models also suffer from stereotypical bias, where algorithms learn spurious correlations between demographic attributes and specific emotions [5], [20].”