Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches

Probably not a substantial claim: may not pass S4 (based).

The COMPAS debates have helped to entrench statistical notions of fairness into a range of ML fairness research efforts.

“The COMPAS debates have helped to entrench statistical notions of fairness into a range of ML fairness research efforts.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

ML fairness is a discourse that is deeply embedded within larger institutional and social arrangements (structures) of power that actively shape our shared social world.

“As this article will argue, ML fairness is a discourse that is deeply embedded within larger institutional and social arrangements (structures) of power that actively shape our shared social world.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Improving the accuracy and thus fairness of algorithmic systems often involves surveillance without people's consent, as well as the exploited labor of people who manually classify objects for training purposes.

“Others have critiqued the ways that improving the accuracy and thus”

• cites Kulluri, 2020Not checked yet

  Not checked in the free run.

• cites Sloane et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

These debates do not adequately capture the historical roots of contemporary institutional aspirations for algorithmic fairness.

“Yet, these debates do not adequately capture the historical roots of contemporary institutional aspirations for algorithmic fairness.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

There are frequent conflicts between fairness and profit maximization, and service providers are often focused on outcomes rather than processes.

“There are frequent conflicts between fairness and profit maximization, and service providers are often focused on outcomes rather than processes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some scholars have called for greater attention to the ethics of data collection underpinning ML fairness research, including the ways that cultural values and social and historical contingencies get embedded into data collection, cleaning, curating, and management.

“Some scholars have also called for greater attention to the ethics of data collection underpinning ML fairness research, including the ways that cultural values and social and historical contingencies get embedded into data collection, cleaning, curating, and management (Elish & boyd, 2017).”

• cites Elish & boyd, 2017Not checked yet

  Not checked in the free run.

May combine two claims.

AI bias is intrinsically tied to the predatory inclusion of marginalized groups into AI systems, because attempts to debias AI can be predicated on unethical data collection practices and/or expose marginalized groups to increased surveillance.

“The issue of AI bias is intrinsically tied to the issue of the predatory inclusion of marginalized groups into AI systems, given the ways that attempts to debias AI can be predicated on unethical data collection practices and/or expose marginalized groups to increased surveillance (West et al., 2019;Bennett & Keyes, 2019).”

• cites West et al., 2019Not checked yet

  Not checked in the free run.

• cites Bennett & Keyes, 2019Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Racial categorization is inseparable from the systems of segregation and stratification in housing, education, employment, and civic life that allow for racially marked individuals to be sorted and then subject to disparate opportunities and outcomes.

“Racial categorization is inseparable from the systems of segregation and stratification in housing, education, employment, and civic life that allow for racially marked individuals to be sorted and then”

• cites Benthall and Haynes (2019)Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Critical race theory interrogates how interlocking oppressions are embedded in purportedly color-blind laws and institutions.

“critical race theory interrogates how these interlocking oppressions are embedded in purportedly color-blind laws and institutions (Crenshaw, 2019)”

• cites Crenshaw, 2019Not checked yet

  Not checked in the free run.

May combine two claims.

The structural legacies of colonialism include exploitative medical and scientific experimentation on colonized peoples, the appropriation of land and labor, and control over social structures in order to oppress colonized populations.

“These legacies include exploitative medical and scientific experimentation on colonized peoples, the appropriation of land and labor, and control over social structures in order to oppress colonized populations (Mohamed et al. 2020, see also Rodney, 2018).”

• cites Mohamed et al. 2020Not checked yet

  Not checked in the free run.

• cites Rodney, 2018Not checked yet

  Not checked in the free run.

May combine two claims.

The explicit goal of the NSF program is to fund computational research that will promote the acceptance of AI systems, and AI's capacities for social good are assumed a priori.

“Thus, the explicit goal of this NSF program is to fund computational research that will promote the acceptance of AI systems, and AI's capacities for social good are assumed a priori.”

Classified as probably not a substantial claim.

This fixation on parity or imparity between racial groups overlooks fairness considerations that cannot be captured statistically, such as whether the tool's fundamental purpose is itself fair.

“This fixation on parity or imparity between racial groups overlooks fairness considerations that cannot be captured statistically, such as whether the tool's fundamental purpose is itself fair (Green & Hu, 2018).”

• cites Green & Hu, 2018Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Different potential proxies frame a given problem in subtly different ways and raise different ethical concerns.

“Different potential proxies frame a given problem in subtly different ways and raise different ethical concerns.”

Classified as probably not a substantial claim.

A given AI tool's application may be free from bias in a statistical sense, but this does not mean it is fair in an ethical sense.

“Ultimately, while a given AI tool's application may be free from bias in a statistical sense, this does not mean it is fair in an ethical sense (Mitchell et al., 2021).”

• cites Mitchell et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

In many cases, data annotation work is done by prisoners and the economically vulnerable, and in geographic areas with limited worker protections.

“In many cases, this work is done by prisoners and the economically vulnerable, and in geographic areas with limited worker protections.”

• cites Mohamed et al. 2020Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The author sees this assumption as a feature of the ways that power dynamics shape knowledge claims, the research that gets funded and given access, and finally, who generally gets to conduct this research.

“The author sees this assumption as a feature of the ways that power dynamics shape knowledge claims, the research that gets funded and given access, and finally, who generally gets to conduct this research.”

Classified as probably not a substantial claim.

This paper points to the limitations of training data and legally protected categories for doing justice to complex and highly contingent and contested fairness considerations.

“This paper points to the limitations of training data and legally protected categories for doing justice to complex and highly contingent and contested fairness considerations.”

Probably not a substantial claim: may not pass S2 (checkable).

There remains a risk of checklists being privileged over other efforts, such as regulation, precisely because they do not compel the redistribution of power in the design and implementation of AI tools.

“Furthermore, there remains the risk of checklists being privileged over other efforts, such as regulation, precisely because they do not compel the redistribution of power in the design and implementation of AI tools.”

Classified as probably not a substantial claim. May combine two claims.

Goel et al.'s article, while showing ML research that grapples with systemic discrimination and evaluates dominant institutions of power, does not sufficiently engage with the larger sociotechnical context of US policing, including the criminalization of poverty and the structural conditions that produce criminogenic factors.

“it does not sufficiently engage with the larger sociotechnical context of US policing, including the criminalization of poverty and the structural conditions that produce criminogenic factors in the first place.”

• cites Goel et al. (2016)Not checked yet

  Not checked in the free run.

May combine two claims.

Images of students, immigrants, abused children, people who have had mugshots taken, and deceased people have all been used without consent to test facial recognition technology within and outside of the US.

“Furthermore, images of students, immigrants, abused children, people who have had mugshots taken, and deceased people have all been used without consent to test facial recognition technology within and outside of the US (e.g., Keyes et al., 2019;Harvey & LaPlace, 2019).”

• cites Keyes et al., 2019Not checked yet

  Not checked in the free run.

• cites Harvey & LaPlace, 2019Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The uncritical use of COMPAS to establish the efficacy of Foulds et al.'s differential fairness classifier method speaks to the limitations of this approach.

“While the paper nuances discussions of fairness with a greater consideration of how different systems of oppression intersect, the uncritical use of COMPAS to establish the efficacy of their methods speaks to the limitations of this approach.”

• cites Foulds, Islam, Keya, and Pan (2018)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Predictive policing derives data from communities that are disproportionately patrolled, creating a feedback loop that intensifies the surveillance of these communities.

“One such example they provide is of predictive policing, wherein data is derived from communities that are disproportionately patrolled, creating a feedback loop that intensifies the surveillance of these communities.”

• cites Overdorf et al., 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Not all solutions are equally well positioned to fundamentally disrupt the existing power dynamics and structural injustices historically and presently shaping the field.

“However, not all solutions are equally well positioned to fundamentally disrupt the existing power dynamics and structural injustices historically and presently shaping the field.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Sloane et al. (2020) have cautioned against 'participation-washing,' wherein powerful agencies use the participation of communities to manufacture consent and legitimize injustice.

“Sloane et al. (2020) have also cautioned against”

• cites Sloane et al., 2020Not checked yet

  Not checked in the free run.

May combine two claims.

The limitation that causal graphs do not address power imbalances also applies to Google's ML-fairness gym, which does not require AI researchers to work with community members or domain experts from other disciplines who might be better positioned to understand how agents' behaviors are institutionally conditioned and constrained by power dynamics within a given social context.

“On the other hand, however, it does not require that AI researchers work with community members or domain experts from other disciplines who might be better positioned to understand how agents' behaviors are institutionally conditioned and constrained by power dynamics within a given social context.”

• cites D'Amour et al., 2020Not checked yet

  Not checked in the free run.

May combine two claims.

The term 'unintended consequences' can function as a defensive and dismissive strategy that allows for the abdication of developer responsibility and accountability, and a lack of consideration of the complexity of social systems in such a way as to lead to quick technical fixes.

“Thus, the term can function as a defensive and dismissive strategy that allows for the abdication of developer responsibility and accountability, on the one hand, and a lack of”

Classified as probably not a substantial claim.

A variety of solutions have been proposed for addressing the limits and harms of hegemonic ML fairness research.

“A variety of solutions have been proposed for addressing the limits and harms of hegemonic ML fairness research described above.”

May combine two claims.

Improving the efficiency and accuracy of algorithms for pre-trial risk assessment reinforces carceral logics and outcomes, and siphons off political will for more radical and necessary reforms, such as the ending of cash bail and pretrial detention.

“One such example is improving the efficiency and accuracy of algorithms for pre-trial risk assessment, which reinforces carceral logics and outcomes, and siphons off political will for more radical and necessary reforms, such as the ending of cash bail and pretrial detention (Green, 2019).”

• cites Green, 2019Not checked yet

  Not checked in the free run.

May combine two claims.

Datasheets, on their own, do not create mechanisms for addressing the power asymmetries that exist at all stages of AI's design, development, and deployment, nor necessarily curtail unethical data collection practices.

“Datasheets, on their own, do not create mechanisms for addressing the power asymmetries that exist at all stages of AI's design, development, and deployment, nor necessarily curtail unethical data collection practices, but they could certainly be coupled with these efforts.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Such ML research might support both technical and non-technical solutions, and empower groups working to build a more just and equitable world.

“Such ML research might support both technical and non-technical solutions, and empower groups working to build a more just and equitable world.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

In order for participatory design to distinguish itself from participation-washing, those from marginalized groups who are subject to heightened levels of scrutiny and potential harm within many AI systems must have a significant degree of control over the development, deployment, and assessment of the AI tool.

“It is important to note that in order for participatory design to distinguish itself from participation-washing, those from marginalized groups who are subject to heightened levels of scrutiny and potential harm within many AI systems must have a significant degree of control over the development, deployment, and assessment of the AI tool.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The more recent European AI Act has received similar challenges concerning its efficacy.

“The more recent European AI Act (European Commission, 2021), a risk-based framework for regulating AI, has received similar challenges concerning its efficacy.”

• cites European Commission, 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

It remains possible to imagine design processes for AI tools that are explicitly geared towards anti-oppressive ends.

“While Gangadharan & Niklas (2019) raise an essential point about the constraints of techno-centrism, it remains possible to imagine design processes for AI tools that are explicitly geared towards anti-oppressive ends.”

Classified as probably not a substantial claim. May combine two claims.

Most fairness, accountability, transparency, and data justice studies approaches are techno-centric and process-oriented, when what is needed is an evaluative ethical framework that decenters technology.

“However, Gangadharan and Niklas (2019) argue that most fairness, accountability, transparency, and data justice studies approaches are techno-centric and process-oriented, when what is needed is an evaluative ethical framework that decenters technology.”

• cites Gangadharan and Niklas (2019)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Selbst et al.'s (2019) solution would hinge on the degree to which marginalized people are involved and empowered.

“Selbst et al.'s (2019) solution is explicitly proposed to mitigate technological solutionism and exclusionary AI design practices, and this would hinge on the degree to which marginalized people are involved and empowered.”

• cites Selbst et al. (2019)Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

An evaluative ethical framework that decenters technology would position sociotechnical systems of discrimination alongside other modalities and offer nuance into what injustice discrimination causes, who is affected, and how discrimination can be remedied.

“According to the authors, such an approach would”

• cites Gangadharan & Niklas, 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

There is reason to believe that interdisciplinary collaborations would disrupt existing institutionalized power dynamics within ML fairness research.

“Given the ways that interdisciplinary collaborations have the potential to challenge hegemonic computational epistemologies and practices, and particularly, problems of technological solutionism and abstraction, there is reason to believe that such collaborations would disrupt existing institutionalized power dynamics within ML fairness research.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Hanna et al.'s (2020) approach might be appropriated within the ML fairness community to function as a more elaborate discussion of the limits of a chosen classification system, without curtailing harmful ML research.

“However, it is also possible that, like Elish and boyd's (2017) recommendation for more self-reflexive research practices, such an approach might be appropriated within the ML fairness community to function as a more elaborate discussion of the limits of a chosen classification system, without curtailing harmful ML research.”

• cites Hanna et al. (2020)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Acts of modifying, subverting, or sabotaging an optimization system can also be used for antisocial ends.

“Additionally, while POTs can be used to help users avoid or disrupt harmful optimization systems, acts of modifying, subverting, or sabotaging an optimization system can also be used for antisocial ends.”

Probably not a substantial claim: may not pass S2 (checkable).

Grassroots coalition-based movements, ideally bolstered by the support of AI fairness researchers, would be essential for pushing for the disruption of entrenched institutional power dynamics.

“This means that grassroots coalitionbased movements, ideally bolstered by the support of AI fairness researchers, would be essential for pushing for the disruption of entrenched institutional power dynamics.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

This work necessarily requires de-centering Western notions of algorithmic fairness in non-Western contexts, including working with relevant culturally and historically contextualized categories, social structures, power dynamics, value-systems, and going beyond model fairness.

“This work necessarily requires de-centering Western notions of algorithmic fairness in non-Western contexts, including working with relevant culturally and historically contextualized categories, social structures, power dynamics, value-systems, and going beyond model fairness (Sambasivan et al., 2021).”

• cites Sambasivan et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

While regulation has the potential to disrupt existing power dynamics, regulatory frameworks can also be used to further consolidate and centralize power, assets, control, and/or rights in the hands of a minority.

“However, while regulation has the potential to disrupt existing power dynamics, regulatory frameworks can also be used to further consolidate and centralize power, assets, control, and/or rights in the hands of a minority, such as in the context of international AI governance standards and AI for international social development that often privilege Western interests (Mohamed et al., 2020;Sambasiv”

• cites Mohamed et al., 2020Not checked yet

  Not checked in the free run.

• cites Sambasivan et al. 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

It is not specified to what degree the participation of impacted communities figures into Umbrello and van de Poel's (2021) approach, nor whether these approaches set out to substantively alter relations of power in society.

“However, it is not specified to what degree the participation of impacted communities figures into this approach, nor whether these approaches set out to substantively alter relations of power in society.”

• cites Umbrello and van de Poel's (2021)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The European Union General Data Protection Regulation has been criticized as being too slow and too soft.

“Furthermore, regulations like the European Union General Data Protection Regulation (2016), a series of data protection rules implemented in 2018 for addressing data security and issues of consent, transparency, and accountability, have been criticized as being”

• cites Bodoni, 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Over one hundred civil society organizations have argued in a collective statement that the European AI Act is in need of amendments in order to ensure it is precise and robust enough to address structural imbalances of power.

“Over one hundred civil society organizations have argued in a collective statement that the act is in need of amendments in order to ensure it is precise and robust enough to address structural imbalances of power (”

• cites An EU Artificial Intelligence Act, 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Redesigning AI tools with the full participation of people who are intersectionally disadvantaged under capitalism, white supremacy, heteropatriarchy, and colonialism, and incorporating intersectional feminist thinking into the design and evaluation of technical standards and bias audits, could improve AI outcomes for marginalized groups in contexts where data gaps for minoritized people are系统地贡献于

“The redesigning of AI tools with the full participation of people who are intersectionally disadvantaged under capitalism, white supremacy, heteropatriarchy, and colonialism, and the meaningful incorporation of intersectional feminist thinking into the design and evaluation of technical standards and bias audits, could improve AI outcomes for marginalized groups in contexts where data gaps for min”

• cites Costanza-Chock, 2018Not checked yet

  Not checked in the free run.

• cites Adamson & Smith, 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Social pressure through organized movements, strong reward systems for technology that empowers the least privileged, and a completely new application of technologies are best positioned to counter the ways that the AI/ML community's current incentive structures pressure even the most well-intentioned developers towards maintaining existing power dynamics.

“In addition to stricter regulation, Birhane (2021) argues that,”

• cites Birhane (2021)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Such an approach would make it possible for the design process to meaningfully leverage the knowledge that comes from people who occupy subjugated standpoints in society.

“Such an approach would make it possible for the design process to meaningfully leverage the knowledge that comes from people who occupy subjugated standpoints in society (Costanza-Chock, 2020).”

• cites Costanza-Chock, 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Participatory design processes that reimagine power relations between technologists and marginalized communities have the potential to significantly address what is most ethically concerning rather than what is most convenient to measure.

“Participatory design processes that reimagine power relations between technologists and marginalized communities have the potential to significantly address what is most ethically concerning rather than what is most convenient to measure (Blodgett et al., 2020).”

• cites Blodgett et al., 2020Not checked yet

  Not checked in the free run.