CoChat
Sign Up
Cited by 74Ethics and Social Impacts of AIBayesian Modeling and Causal InferenceExplainable Artificial Intelligence (XAI)

What's sex got to do with machine learning?

Open access
Hu, Lily, Kohler-Hausmann, Issa·2020·Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
DOI 10.1145/3351095.3375674
Open this paper in CoChat — free accountOpen PDF

Abstract

The debate about fairness in machine learning has largely centered around competing substantive definitions of what fairness or nondiscrimination between groups requires. However, very little attention has been paid to what precisely a group is. Many recent approaches have abandoned observational, or purely statistical, definitions of fairness in favor of definitions that require one to specify a causal model of the data generating process. The implicit ontological assumption of these exercises is that a racial or sex group is a collection of individuals who share a trait or attribute, for example: the group "female" simply consists in grouping individuals who share female-coded sex features. We show this by exploring the formal assumption of modularity in causal models using directed acyclic graphs (DAGs), which hold that the dependencies captured by one causal pathway are invariant to interventions on any other causal pathways. Modeling sex, for example, as a node in a causal model aimed at elucidating fairness questions proposes two substantive claims: 1) There exists a feature, sex-on-its-own, that is an inherent trait of an individual that then (causally) brings about social phenomena external to it in the world; and 2) the relations between sex and its downstream effects can be modified in whichever ways and the former node would still retain the meaning that sex has in our world. Together, these claims suggest sex to be a category that could be different in its (causal) relations with other features of our social world via hypothetical interventions yet still mean what it means in our world. This fundamental stability of categories and causes (unless explicitly intervened on) is essential in the methodology of causal inference, because without it, causal operations can alter the meaning of a category, fundamentally change how it is situated within a causal diagram, and undermine the validity of any inferences drawn on the diagram as corresponding to any real phenomena in the world.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3351095.3375674

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Even causal explanations may be problematic because they rely on flawed ontological assumptions about the social categories used for classification.

    “Even causal explanations may be problematic, as recent work has shown (Hu & Kohler-Hausman, 2020), because they rely on flawed ontological assumptions about the social categories used for classification.”

    From Algorithmic profiling as a source of hermeneutical injustice (2025) · cites this paper as Hu & Kohler-Hausman, 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 21

  1. [1][1] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference . ACM, 214-226. -
  2. [2][2] Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems , 3315-3323. -
  3. [3][3] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. In Proceedings of the 8th Innovations in Theoretical Computer Science Conference , ACM, 1-43. -
  4. [4][4] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , ACM, 797-806. -
  5. [6][6] Shira Mitchell, Eric Potash, Solon Barocas, Alexander D'Amour, and Kristian Lum. Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions. 2019. arXiv:1811.07867v2 (2019). - · arXiv
  6. [7][7] Sam Corbett-Davies and Sharad Goel. 2018. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. arxiv:1808.00023 (2018). - · arXiv
  7. [8][8] Matt J. Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. In Advances in Neural Information Processing Systems , ACM, 4066-4076. -
  8. [9][9] Razieh Nabi and Ilya Shpitser. 2018. Fair inference on outcomes. In Proceedings of the 32 nd AAAI Conference on Artificial Intelligence , 1931-1940. -
  9. [11][11] Junzhe Zhang and Elias Bareinboim. 2018. Fairness in decision-making-the causal explanation formula. In 32nd AAAI Conference on Artificial Intelligence , 2037-2045. -
  10. [12][12] Judea Pearl. 2009. Causality: Models, Reasoning, and Inference (2nd. ed.) . Cambridge University Press, Cambridge, United Kingdom. -
  11. [14][14] Peter J. Bickel, Eugene A. Hammel, and J. William O'Connell. 1975. Sex bias in graduate admissions: Data from Berkeley. Science, 187, 4175, (February 1975), 398-404. DOI: 10.1126/science.187.4175.398. - · DOI · Paper page
  12. [15][15] Daniel M. Hausman and James Woodward. 1999. Independence, Invariance, and the Causal Markov Condition. The British Journal for the Philosophy of Science 50, 4, (December 1999), 521-583. DOI: https://doi.org/10.1093/bjps/50.4.521 - · DOI
  13. [17][17] Clark Glymour. 1986. Comment: Statistics and Metaphysics. Journal of the American Statistical Association 81, 396, (December 1986), 964-966. DOI: https://doi.org/10.1080/01621459.1986.10478357 - · DOI
  14. [18][18] Clark Glymour and Madelyn R. Glymour. 2014. Commentary: Race and Sex Are Causes. Epidemiology 25, 4, (July 2014), 488-490. DOI: https://doi.org/10.1097/EDE.0000000000000122 - · DOI
  15. [19][19] Peter Sprites, Clark Glymour, and Richard Scheines. 2001. Causation, Prediction, and Search (2nd. Ed.). MIT Press, Cambridge, MA -
  16. [20][20] James Woodward. 2003. Making Things Happen: A Theory of Causal Explanation , Oxford University Press, Oxford, United Kingdom. -
  17. [22][22] Nancy Cartwright. 2007. Hunting Causes and Using Them: Approaches in Philosophy and Economics . Cambridge University Press, Cambridge, United Kingdom. -
  18. [24][24] Sally Haslanger. 2012. Resisting Reality: Social Construction and Social Critique. Oxford University Press, Oxford, United Kingdom -
  19. [25][25] Ian Hacking. 1999. The Social Construction of What? Harvard University Press, Cambridge, MA -
  20. [26][26] Judith Butler. 1988. Performative Acts and Gender Constitution: An Essay in Phenomenology and Feminist Theory, Theatre Journal , 40, 4, (December 1988), 519-531. DOI:10.2307/3207893 - · DOI
  21. [28][28] Issa Kohler-Hausmann. 2019 . Eddie Murphy and the Dangers of Counterfactual Causal Thinking About Detecting Racial Discrimination. Northwestern University Law Review , 113, 5, 1163-1228, 1179.

Cited by

Sorted by citations

Authors
Hu, L., & Kohler-Hausmann, I.
Affiliations
  • Harvard University (US)
  • Yale University
ORCID
Hu, Lily
Venue
Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2020-01-27
Pages
513-513
DOI
10.1145/3351095.3375674
OpenAlex
W3013969094
Cited by
74
References
2
Citations by year
2026: 5 · 2025: 5 · 2024: 12 · 2023: 19 · 2022: 11 · 2021: 17 · 2020: 4 · 2019: 1
Type
conference abstract
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
Funders
National Science Foundation Graduate Research Fellowship Program (DGE1745303)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic fairness, causal models, directed acyclic graphs, non-discrimination, sex categories

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review