CoChat
Sign Up
Cited by 709Ethics and Social Impacts of AIFeminist Epistemology and Gender StudiesArtificial Intelligence in Law

Algorithmic injustice: A relational ethics approach

Open access
Birhane, Abeba·2021·Patterns
DOI 10.1016/j.patter.2021.100205PMC PMC7892355
Open this paper in CoChat — free accountOpen PDF

Abstract

It has become trivial to point out that algorithmic systems increasingly pervade the social sphere. Improved efficiency-the hallmark of these systems-drives their mass integration into day-to-day life. However, as a robust body of research in the area of algorithmic injustice shows, algorithmic systems, especially when used to sort and predict social outcomes, are not only inadequate but also perpetuate harm. In particular, a persistent and recurrent trend within the literature indicates that society's most vulnerable are disproportionally impacted. When algorithmic injustice and harm are brought to the fore, most of the solutions on offer (1) revolve around technical solutions and (2) do not center disproportionally impacted communities. This paper proposes a fundamental shift-from rational to relational-in thinking about personhood, data, justice, and everything in between, and places ethics as something that goes above and beyond technical solutions. Outlining the idea of ethics built on the foundations of relationality, this paper calls for a rethinking of justice and ethics as a set of broad, contingent, and fluid concepts and down-to-earth practices that are best viewed as a habit and not a mere methodology for data science. As such, this paper mainly offers critical examinations and reflection and not "solutions."

AI snapshot

Written from the paper’s full text.

TL;DR

Birhane argues that algorithmic injustice is perpetuated by technical fixes that fail to center disproportionately impacted communities, and proposes a shift from rational to relational ethics. Relational ethics grounds justice and ethics in context, history, lived experience, and power asymmetries rather than technical solutions.

Key findings

  • The paper contends that dominant responses to algorithmic harm—such as debiasing datasets and fairness metrics—are inadequate because they treat bias as a static deviation from a “correct” description and obscure structural, historical, and power asymmetries.
  • It proposes four tenets of relational ethics: knowing that centers human relations and the disproportionately impacted; bias is not a deviation from the “correct” description; prioritizing understanding over prediction; and treating data science as a practice that alters the social fabric.
  • It claims marginalized groups are disproportionately harmed by algorithmic systems and hold epistemic privilege to recognize harm, citing examples including object detection, recidivism, gender recognition/classification, and medicine.
  • It calls for ethics as a down-to-earth habit and practice, not a mere methodology or tool, and states that this work is never done.

Methods

  • Design: critical, conceptual, and reflective essay; no empirical study, dataset, or sample is used.
  • Main analysis: comparative and philosophical argument drawing on Afro-feminist epistemology, enactive cognitive science, ubuntu, Bakhtinian dialogism, and complexity science to contrast rationality and relationality.
  • No quantitative analysis or original numerical results are reported.

Limitations

  • The paper states it mainly offers critical examinations and reflection and not “solutions”; it is not a set of general guidelines, principles, or out-of-the-box tools.
  • The paper states relational ethics is not a rationally constructed abstract “theory of ethics” and does not provide implementable remedies.
  • Evident from the text: no empirical validation, case study, or measurement is provided; the framework is conceptual and programmatic.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.patter.2021.100205

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 64 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Afro-feminist epistemology grounds knowing in an active and engaged practice.

    “Pushing back against the dominant Western orthodoxy, Afro-feminist epistemology grounds knowing in an active and engaged practice.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • Premisecitation

    The AI industry, grounded in rationality, reproduces harmful and discriminatory outcomes.

    “Mhlambi11 has further argued that the AI industry, grounded in rationality, reproduces harmful and discriminatory outcomes.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] Mhlambi, 2020, From rationality to relationality: ubuntu as an ethical and human rights framework for artificial intelligence governance

      Not checked in the free run.

  • Premisecitation

    The reality of the Western straight white male masquerades as the invisible background that is taken as the normal, standard, or universal position.

    “The reality of the Western straight white male then masquerades as the invisible background that is taken as the “normal,” “standard,” or “universal” position.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Ahmed, 2007, A phenomenology of whiteness, Femin. Theor. source

      Not checked in the free run.

  • Premisecitation

    Society's most vulnerable are disproportionately impacted by algorithmic injustice and harm.

    “a persistent and recurrent trend within the literature indicates that society's most vulnerable are disproportionally impacted.”
    Quote in the paperSubstantial claim
  • Premisecitation

    For Afro-feminist thought, the researcher is an important participant in the knowledge production process.

    “In contrast, for Afro-feminist thought, the researcher is an important participant in the knowledge production process.33”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Nnaemeka, 2004, Nego-feminism: theorizing, practicing, and pruning Africa’s way, Signs source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 62
  • May combine two claims. 19

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    In the process of data collection, the data scientist decides what is worth measuring, making some things visible and others invisible by default, and how.

    “In the process of data collection, for example, the data scientist decides what is worth measuring (making some things visible and others invisible by default) and how.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Wisdom, and not book learning, enables one to resist oppression.

    “Wisdom, and not “book learning,” enables one to resist oppression.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    The rational worldview that aspires to an objective knowledge from a God's eye view has resulted in the treatment of the researcher as invisible, and their interests, motivations, and background as inconsequential.

    “The rational worldview that aspires to an “objective” knowledge from a “God's eye view” has resulted in the treatment of the researcher as invisible, and their interests, motivations, and background as inconsequential.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    In the desire to establish timeless and absolute knowledge, abstract and contextless reasoning is prioritized over concrete lived experience submersed in co-relations, interdependence, fluidity, and connectedness.

    “In the desire to establish timeless and absolute knowledge, abstract and contextless reasoning is prioritized over concrete lived experience submersed in co-relations, interdependence, fluidity, and connectedness.16”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] Merleau-Ponty, 1968

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Knowledge claims are not worked out in isolation from others but are developed in dialog with the community.

    “Knowledge claims are not worked out in isolation from others but are developed in dialog with the community.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisenone given

    Relational frameworks emphasize the primacy of relations and dependencies.

    “Relational frameworks emphasize the primacy of relations and dependencies.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Knowledge, according to the rational worldview, is rooted in the ideal rational, static, self-contained, and self-sufficient subject that contemplates the external world from afar in a purely cognitive manner as a disembodied and disinterested observer.

    “Knowledge, according to this worldview, is rooted in the ideal rational, static, self-contained, and self-sufficient subject that contemplates the external world from afar in a “purely cognitive” manner as a disembodied and disinterested observer.15”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [15] Gardiner, 1998, The incomparable monster of solipsism: Bakhtin and Merleau-Ponty

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Knowing and being are active processes that are necessarily political and ethical.

    “Knowing and being are active processes that are necessarily political and ethical.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Algorithmic systems, especially when used to sort and predict social outcomes, are not only inadequate but also perpetuate harm.

    “algorithmic systems, especially when used to sort and predict social outcomes, are not only inadequate but also perpetuate harm.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Bayesian models are prone to spurious relationship and amplification of socially held stereotypes.

    “However, despite the association of Bayes with rational predictions, Bayesian models are prone to spurious relationship and amplification of socially held stereotypes.24”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] Pager, 2009, Bayesian bigot? Statistical discrimination, stereotypes, and employer decision making, Ann. Am. Acad. Polit. Soc. Sci. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Discussions and understanding of concepts such as knowledge and ethics tend to be abstract, genderless, contextless, and raceless.

    “Subsequently, discussions and understanding of concepts such as knowledge and ethics tend to be abstract, genderless, contextless, and raceless.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    The rational worldview holds that reason and logical coherence are superior for knowledge production above and beyond relational and embodied becoming.

    “can be exemplified by the deep contention that reason and logical coherence are superior for knowledge production (in understanding the world) above and beyond relational and embodied becoming.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    Since the rational worldview has come to be seen as the standard, anything outside of this is viewed as an outlier.

    “Since the rational worldview has come to be seen as the standard, anything outside of this is viewed as an outlier.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Our most sophisticated human knowing lies in how we engage with each other.

    “A proponent of this position, Hanne De Jaegher,35 contends that our most sophisticated human knowing lies in how we engage with each other.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] De Jaegher, 2019, Loving and knowing: reflections for an engaged epistemology, Phenomenol. Cogn. Sci.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    The privileging of reason as the ultimate criterion makes knowing a distant act.

    “The privileging of reason as the ultimate criterion makes knowing a distant act.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Knowing is an activity that happens in the relationship between the knower and the known.

    “Far from a distant and “objective” discretizing logic, knowing is an activity that happens in the relationship between the knower and the known.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] De Jaegher, 2019, Loving and knowing: reflections for an engaged epistemology, Phenomenol. Cogn. Sci.

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    The self is social through and through-it is co-generated in interactions and relations with others.

    “Kyselo,27 for example, contends that the self is social through and through—it is co-generated in interactions and relations with others.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] Kyselo, 2014, The body social: an enactive approach to the self, Front. Psychol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Current data practices, for the most part, follow the rational model of thinking where data are assumed to represent the world out there in a neutral way.

    “Current data practices, for the most part, follow the rational model of thinking where data are assumed to represent the world out there in a “neutral” way.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    In the process of data cleaning, rich information that provides context about which data are collected and how datasets are structured is stripped away.

    “In the process of data cleaning, rich information that provides context about which data are collected and how datasets are structured is stripped away.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    All bodies inherit history and the inheritance of Cartesianism is grounded in a white straight ontology.

    “Ahmed17 contends that all bodies inherit history and the inheritance of Cartesianism is grounded in a white straight ontology.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Ahmed, 2007, A phenomenology of whiteness, Femin. Theor. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Any data scientist working to automate issues of a social nature is engaged in making moral and ethical decisions.

    “Any data scientist working to automate issues of a social nature, in effect, is engaged in making moral and ethical decisions”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The rational view serves as the backbone for much of Western science and philosophy, permeating most fields of enquiry.

    “The rational view serves as the backbone for much of Western science and philosophy, permeating most fields of enquiry”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [11] Mhlambi, 2020, From rationality to relationality: ubuntu as an ethical and human rights framework for artificial intelligence governance

      Not checked in the free run.

    • cites [12] Terry, 1986

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Nothing is simply itself outside the matrix of relations in which it exists.

    “Bakhtin29 emphasized that nothing is simply itself outside the matrix of relations in which it exists.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Bakhtin, 1984

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The most reliable form of knowledge, especially concerning social and historical injustice, is grounded in lived experience.

    “The most reliable form of knowledge, especially concerning social and historical injustice, is grounded in lived experience.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Concrete experiences are primary and abstract reasoning secondary.

    “At the heart of it, the Afro-feminist approach to knowing contends that concrete experiences are primary and abstract reasoning secondary.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The mathematization and formalization of social issues brings a veneer of objectivity and positions its operations as value-free, neutral, and amoral.

    “The mathematization and formalization of social issues brings with it a veneer of objectivity and positions its operations as value-free, neutral, and amoral.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Bowker, 2000

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Data science and data practices reincarnate rationalism in many forms, including intolerance of messiness, ambiguity, and uncertainty.

    “Data science and data practices reincarnate rationalism in many forms, including in the manner in which messiness, ambiguity, and uncertainty are not tolerated”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Distant statistics or theoretical accuracies do not take precedence over the actual experience of a person.

    “Distant statistics or theoretical accuracies do not take precedence over the actual experience of a person.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Collins, 2002

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    The deep quest for the rational worldview is certainty, stability, and order, and thus isolation, separation, and clear binaries form the foundations in place of connectedness, interdependence, and dynamic relation.

    “The deep quest for the rational worldview is certainty, stability, and order, and thus isolation, separation, and clear binaries form the foundations in place of connectedness, interdependence, and dynamic relation.13”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Prigogine, 1984

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Selfhood and knowledge are evolving and dynamic; the self is never finished.

    “Selfhood and knowledge are evolving and dynamic; the self is never finished—it is an open book.30”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Birhane, 2017, Descartes was wrong:‘a person is a person through other persons’, Aeon

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    The inheritance of rationality to data sciences and computation makes these fields inadequate to deal with complex and inherently indeterminable phenomena.

    “Not only does the inheritance of rationality to data sciences and computation make these fields inadequate to deal with complex and inherently indeterminable phenomena”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Relational perspectives view existence as fundamentally co-existent in a web of relations.

    “Contrary to the rationalist and individualist worldview, relational perspectives view existence as fundamentally co-existent in a web of relations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Any generalization or theory constructed in the absence of deep understanding, not grounded in the concrete and particular, is vacuous.

    “Any generalization or theory constructed in the absence of deep understanding, not grounded in the concrete and particular, is vacuous.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] Geertz, 1973

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Machine learning presents an extreme case of a very human predicament: the only way we can generalize is on the basis of past experience.

    “machine learning presents an extreme case of a very human predicament, which is that the only way we can generalize is on the basis of past experience.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [61] Gross

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Bias, injustice, and discrimination are misconceived as being able to be permanently corrected.

    “In a supposedly objective worldview, bias, injustice, and discrimination are (mis)conceived as being able to be permanently corrected.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Gender classification systems often operate under essentialist assumptions and operationalize gender in a trans-exclusive way, resulting in disproportionate harm to trans people.

    “Furthermore, gender classification systems often operate under essentialist assumptions and operationalize gender in a trans-exclusive way, resulting in disproportionate harm to trans people.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [48] Hamidi, 2018, Gender recognition or gender reductionism? The social implications of embedded gender recognition systems

      Not checked in the free run.

    • cites [50] Keyes, 2018, The misgendering machines: trans/HCI implications of automatic gender recognition, Proc. ACM Hum.-Comput. Interact.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Eventness is always particular, and never exhaustively describable in terms of rules.

    “Eventness is always particular, and never exhaustively describable in terms of rules.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] Morson, 1989

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The machine-detected patterns often provide answers to fuzzy, contingent, and open-ended questions. These answers neither reveal any causal relations nor provide explanation on why or how.

    “The machine-detected patterns often provide “answers” to fuzzy, contingent, and open-ended questions. These “answers” neither reveal any causal relations nor provide explanation on why or how.65”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [65] Pasquale, 2015

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Our deepest intuitions about justice and injustice can change dramatically.

    “our deepest intuitions about all sorts of things, and in particular justice and injustice, can change dramatically.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [61] Gross

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    For any individual person, community, or situation, algorithmic classifications and predictions give either an advantage or they hinder.

    “For any individual person, community, or situation, algorithmic classifications and predictions give either an advantage or they hinder.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisereasoning

    Humans are inherently historical, social, cultural, gendered, politicized, and contextualized organisms.

    “Humans are inherently historical, social, cultural, gendered, politicized, and contextualized organisms.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    In knowing, like loving, what happens is not neutral, general, or universal.

    “Proposing an understanding of human knowing in analogy with loving, De Jaegher argues that in knowing, like loving, what happens is not neutral, general, or universal.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [35] De Jaegher, 2019, Loving and knowing: reflections for an engaged epistemology, Phenomenol. Cogn. Sci.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    Neither people nor the environment and context in which they are embedded are static.

    “neither people nor the environment and context in which they are embedded are static.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The rationalist tradition's tendency toward timeless and generalizable knowledge aspires to establish timeless laws and generalizable theories.

    “The rationalist tradition's tendency toward timeless and generalizable knowledge aspires to establish timeless laws and generalizable theories.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Machine-learning systems detect patterns and cluster similarities, and also make predictions based on the observed patterns.

    “Not only do machine-learning systems detect patterns and cluster similarities, they also make predictions based on the observed patterns.62”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] O’Neil, 2013

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Premisenone given

    The relational versus rational divide is not something that can be clearly demarcated, but overlaps with fuzzy boundaries.

    “Nonetheless, the relational versus rational divide is not something that can be clearly demarcated, but overlaps with fuzzy boundaries.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Daston contends that the endeavor for a universal law is a predicament that does not stand against unanticipated particulars, since no universal ever fits the particulars.

    “Likewise, the historian of science Lorraine Daston contends that the endeavor for a universal law is a predicament that does not stand against unanticipated particulars, since no universal ever fits the particulars.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [60] Daston, 2018, Calculation and the division of labor, 1750-1950, Bull. German Hist. Inst.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    Since knowing is a relational affair, it matters who enters into the knower-known relations.

    “Since knowing is a relational affair, it matters who enters into the knower-known relations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Machine-learning systems work by identifying patterns in vast amounts of data.

    “Machine-learning systems “work” by identifying patterns in vast amounts of data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    What society deems fair and ethical changes over time and with context and culture.

    “What society deems fair and ethical changes over time and with context and culture.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    To understand people, we must take into account unrepeatable contextual meaning.

    “To understand people, we must take into account “unrepeatable contextual meaning.””
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] Morson, 1989

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Living bodies are processes, practices, and networks of relations which have more in common with hurricanes than with statues.

    “Living bodies, according to Di Paolo et al.,37 are processes, practices, and networks of relations which have “more in common with hurricanes than with statues.””
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [37] Di Paolo, 2018

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Machine learning, at its core, is a tool that predicts. It reveals statistical correlations with no understanding of causal mechanisms.

    “Machine learning, at its core, is a tool that predicts. It reveals statistical correlations with no understanding of causal mechanisms.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Human-centered and participatory design approaches often neglect those at the margins and shy away from power asymmetries and structural inequalities.

    “While the latter approaches often neglect those at the margins53 and shy away from power asymmetries and structural inequalities that permeate the social world”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [53] Harrington, 2020, The forgotten margins: what is community-based participatory health design telling us?, Interactions source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Object detection systems designed to predict pedestrians display higher error rates identifying dark-skinned pedestrians while light-skinned pedestrians are identified with higher precision.

    “The findings of Wilson et.al.,44 for instance, demonstrate that object detection systems designed to predict pedestrians display higher error rates identifying dark-skinned pedestrians while light-skinned pedestrians are identified with higher precision.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Wilson, 2019, Predictive inequity in object detection, arXiv

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    Human knowing and understanding of the world necessarily takes place through their respective lenses.

    “Accordingly, their knowing and understanding of the world around them necessarily takes place through their respective lenses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The common belief that bias can be fixed relegates deeply rooted societal and historical injustices, power asymmetries, and structural inequalities to mere datasets.

    “Although datasets are often part of the problem, this commonly held belief relegates deeply rooted societal and historical injustices, nuanced power asymmetries, and structural inequalities to mere datasets.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Concepts such as bias, fairness, and justice are moving targets.

    “Concepts such as bias, fairness, and justice, however, are moving targets.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Underlying the idea of fixing bias is the assumption that there exists a single correct description of reality where a deviation from it has resulted in bias.

    “Furthermore, underlying the idea of “fixing” bias is the assumption that there exists a single correct description of reality where a deviation from it has resulted in bias.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Our most sophisticated knowing is full of uncertainty, inconsistencies, and ambiguities.

    ““Our most sophisticated knowing,” according to De Jaegher, “is full of uncertainty, inconsistencies, and ambiguities.””
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [35] De Jaegher, 2019, Loving and knowing: reflections for an engaged epistemology, Phenomenol. Cogn. Sci.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Premisenone given

    While the rational worldview tends to see knowledge, people, and reality in general as stable, for relational perspectives, we are fluid, active, and continually becoming.

    “Before we delve into that, it is worth reemphasizing that while the rational worldview tends to see knowledge, people, and reality in general as stable, for relational perspectives, we are fluid, active, and continually becoming.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Harm, bias, and injustice from algorithmic systems vary and depend on training and validation data, underlying design assumptions, and the specific deployment context, among other factors.

    “The harm, bias, and injustice that emerge from algorithmic systems varies and is dependent on the training and validation data used, the underlying design assumptions, and the specific context in which the system is deployed, among other factors.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Bakhtin argues that attempts to abstract general rules from particulars lose the most essential thing about human activity, the eventness of the event.

    “Bakhtin argues that such attempts to abstract general rules from particulars “loses the most essential thing about human activity, the very thing in which the soul of morality is to be found,” which Bakhtin calls the “eventness” of the event.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [59] Morson, 1989

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    These systems process people's behaviors, actions, and the social world at large.

    “These systems process people's behaviors, actions, and the social world at large.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    The core of what machine-learning systems do can be exemplified as clustering similarities and differences, abstracting commonalities, and detecting patterns.

    “The core of what machine-learning systems do can be exemplified as clustering similarities and differences, abstracting commonalities, and detecting patterns.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The most marginalized hold the epistemic privilege to recognize harm and injustice.

    “the most marginalized hold the epistemic privilege to recognize harm and injustice”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    Some identities (and not others) are recognized as a pedestrian, or fit for an STEM career, or in need of medical care. Some are ignored and made invisible altogether.

    “Some identities (and not others) are recognized as a pedestrian,44 or fit for an STEM career,69 or in need of medical care.1 Some are ignored and made invisible altogether.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [44] Wilson, 2019, Predictive inequity in object detection, arXiv

      Not checked in the free run.

    • cites [69] Lambrecht, 2019, Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads, Manag. Sci. source

      Not checked in the free run.

    • cites [1] Obermeyer, 2019, Dissecting racial bias in an algorithm used to manage the health of populations, Science source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Seemingly banal tasks such as identifying and predicting employable or criminal characteristics carry grave consequences for those that do not conform to the status quo.

    “Seemingly banal tasks such as identifying and predicting “employable” or “criminal” characteristics carry grave consequences for those that do not conform to the status quo.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The process of categorizing, sorting, and generalizing is far from a mere technical task.

    “The process of categorizing, sorting, and generalizing, therefore, is far from a mere technical task.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Algorithmic systems never emerge in a social, historical, and political vacuum, and to divorce them from the contingent background in which they are embedded is erroneous.

    “Algorithmic systems never emerge in a social, historical, and political vacuum, and to divorce them from the contingent background in which they are embedded is erroneous.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Certain patterns are made visible and types of being objectified while other types are erased.

    “Certain patterns are made visible and types of being objectified while other types are erased.”
    Quote in the paperNot a substantial claim
71 of 71

  • The paper relies on this without support.

    Most solutions on offer for algorithmic injustice revolve around technical solutions.

    “most of the solutions on offer (1) revolve around technical solutions”

  • The paper relies on this without support.

    Most solutions on offer for algorithmic injustice do not center disproportionally impacted communities.

    “and (2) do not center disproportionally impacted communities.”

  • The paper relies on this without support.

    Automated and standardized solutions to complex social issues often contribute more harm than good.

    “automated and standardized solutions to complex and contingent social issues often contribute more harm than good”

  • The paper relies on this without support.

    Algorithmic systems increasingly pervade the social sphere.

    “algorithmic systems increasingly pervade the social sphere.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Concerns over GenAI's sustainability intersect with social justice issues, as communities that already suffer environmental disadvantages stand to be disproportionately harmed.

    “concerns over GenAI's sustainability also intersect with social justice issues, as communities that already suffer environmental disadvantages stand to be disproportionately harmed [16]”

    From Understanding the Practices, Perceptions, and (Dis)Trust of Generative AI among Instructors: A Mixed-methods Study in the U.S. Higher Education (2025) · cites this paper as [16]

    The section “Centering the disproportionally impacted” says that marginalized communities are disproportionately affected by harms from algorithmic systems. It does not connect those harms to GenAI sustainability or identify communities facing environmental disadvantages; searches for environmental and sustainability terms found no passage making that connection. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Birhane (2021) argues that Western AI often imposes rigid, decontextualized systems that clash with the communal and contextual ways knowledge is held in Indigenous cultures.

    “Nevertheless, Birhane (2021) argues that Western AI often imposes rigid, decontextualized systems that clash with the communal and contextual ways knowledge is held in Indigenous cultures.”

    From Data colonialism and indigenous languages in AI: a critical review of existing initiatives and their struggles with data sovereignty (2026) · cites this paper as Birhane 2021

    In “Rationality: the dominant orthodoxy,” Birhane describes Western-influenced data science as treating knowledge as static and context-free. “Knowing that centers human relations” says technologies shaped by a limited Western perspective are forced onto marginalized groups. But the paper discusses Afro-feminist epistemology and ubuntu, not knowledge in Indigenous cultures specifically, so that part of the attribution is unsupported. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • This concentration of agenda-setting power and dominance of a small set of corporate actors risks narrowing the imaginaries of AI futures, privileging commercial interests over broader social concerns.

    “This concentration of agenda-setting power and dominance of a small set of corporate actors risks narrowing the imaginaries of AI futures, privileging commercial interests over broader social concerns [6, 8, 21].”

    From Reimagining Open Source and Openness in AI: Co-Creating Responsible Technological Futures (2026) · cites this paper as [8]

    In “Knowing that centers human relations” (page unavailable), Birhane argues that computing and data science are dominated by privileged groups, limiting the perspectives and concerns reflected in their work. The paper does not establish that a small set of corporate actors controls AI agenda-setting or that commercial interests are privileged over broader social concerns. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The current AI field is shaped by rapid cycles of innovation and hype, with dominant generative AI narratives and agendas largely led by a narrow set of powerful actors who exercise disproportionate influence in how openness is defined, operationalized, and legitimized.

    “The current AI field is shaped by rapid cycles of innovation and hype, with dominant generative AI narratives and agendas largely led by a narrow set of powerful actors (particularly large technology firms, but also influential standards bodies and platform owners) who exercise disproportionate influence in how openness is defined, operationalized, and legitimized.”

    From Reimagining Open Source and Openness in AI: Co-Creating Responsible Technological Futures (2026) · cites this paper as [8]

    Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • AI fairness scholarship has already underscored the need to examine power distribution and structures of exclusion inhabited by AI technologies.

    “AI fairness scholarship has already underscored the need to examine power distribution and structures of exclusion inhabited by AI technologies [15, 93].”

    From AI and Non-Western Art Worlds: Reimagining Critical AI Futures through Artistic Inquiry and Situated Dialogue (2025) · cites this paper as [15]

Cites

Flagged first, then by citations

3 of the 74 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Descartes, 1984, Vol. 2
  2. Birhane, 2017, Descartes was wrong:‘a person is a person through other persons’, Aeon

Cited by

Sorted by citations

Related

Authors
Birhane, A.
Affiliations
University College Dublin (IE)
Corresponding author
Birhane, Abeba
ORCID
Birhane, Abeba
Venue
Patterns
Publisher
Elsevier BV
Published
2021-02
Volume
2
Issue
2
Pages
100205
ISSN
2666-3899 (ISSN-L)
DOI
10.1016/j.patter.2021.100205
PMC
PMC7892355
OpenAlex
W3132771424
Semantic Scholar
a6de042d8a2baeeeb3d7e37e03c66c1fe07e7b36
Cited by
707
References
74
Citations by year
2026: 251 · 2025: 177 · 2024: 85 · 2023: 94 · 2022: 73 · 2021: 25 · 2020: 2
FWCI
12.97
Type
journal article
Language
en
Open access
Licence
Locations
  • Patterns (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Science Foundation Ireland (13/RC/2094_P2)
  • European Regional Development Fund
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Feminist Epistemology and Gender Studies
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Keywords
algorithmic injustice, relational ethics, relationality, algorithmic harms, personhood, data ethics, justice

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review