CoChat
Sign Up
PreprintCited by 16

The DOI matches its DataCite record.

A survey on measuring indirect discrimination in machine learning

Zliobaite, Indre·2015·arXiv
DOI 10.48550/arxiv.1511.00148arXiv 1511.00148
Open this paper in CoChat — free account

Abstract

Nowadays, many decisions are made using predictive models built on historical data.Predictive models may systematically discriminate groups of people even if the computing process is fair and well-intentioned. Discrimination-aware data mining studies how to make predictive models free from discrimination, when historical data, on which they are built, may be biased, incomplete, or even contain past discriminatory decisions. Discrimination refers to disadvantageous treatment of a person based on belonging to a category rather than on individual merit. In this survey we review and organize various discrimination measures that have been used for measuring discrimination in data, as well as in evaluating performance of discrimination-aware predictive models. We also discuss related measures from other disciplines, which have not been used for measuring discrimination, but potentially could be suitable for this purpose. We computationally analyze properties of selected measures. We also review and discuss measuring procedures, and present recommendations for practitioners. The primary target audience is data mining, machine learning, pattern recognition, statistical modeling researchers developing new methods for non-discriminatory predictive modeling. In addition, practitioners and policy makers would use the survey for diagnosing potential discrimination by predictive models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1511.00148

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 35

  1. Kenneth. J. Arrow. 1973. The Theory of Discrimination. In Discrimination in Labor Markets, O. Ashenfelter and A. Rees (Eds.). Princeton University Press, 3-33.
  2. Solon Barocas, Sorelle Friedler, Moritz Hardt, Josh Kroll, Suresh Venkatasubramanian, and Hanna Wallach (Eds.). 2015. 2nd International Workshop on Fairness, Accountability, and Transparency in Machine Learning (FATML). http://www.fatml.org
  3. Solon Barocas and Moritz Hardt (Eds.). 2014. International Workshop on Fairness, Accountability, and Transparency in Machine Learning (FATML). http://www.fatml.org/2014
  4. Solon Barocas and Andrew D. Selbst. 2016. Big Data's Disparate Impact. California Law Review 104 (2016).
  5. Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statis- tics). Springer-Verlag New York, Inc.
  6. Rebecca M. Blank, Marilyn Dabady, Constance Forbes Citro, and National Research Council (U.S.) Panel on Methods for Assessing Discrimination. 2004. Measuring racial discrimination. National Academies Press.
  7. Toon Calders, Asim Karim, Faisal Kamiran, Wasif Ali, and Xiangliang Zhang. 2013. Controlling Attribute Effect in Linear Regression. In Proc. of the 13th Int. Conf. on Data Mining (ICDM). 71-80.
  8. Toon Calders and Sicco Verwer. 2010. Three Naive Bayes Approaches for Discrimination-free Classification. Data Min. Knowl. Discov. 21, 2 (2010), 277-292.
  9. Toon Calders and Indre Zliobaite (Eds.). 2012. IEEE ICDM 2012 International Workshop on Discrimination and Privacy-Aware Data Mining (DPADM). https://sites.google.com/site/dpadm2012/
  10. Danielle K. Citron and Frank A. Pasqualle III. 2014. The Scored Society: Due Process for Automated Pre- dictions. Washington Law Review 89 (2014).
  11. Bart Custers, Toon Calders, Bart Schermer, and Tal Zarsky (Eds.). 2013. Discrimination and Privacy in the Information Society. Springer.
  12. Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In Proc. of the 21th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining. 259-268.
  13. Kazuto Fukuchi, Jun Sakuma, and Toshihiro Kamishima. 2013. Prediction with Model-Based Neutrality. In Proc. of European conference on Machine Learning and Knowledge Discovery in Databases. 499-514.
  14. Isabelle Guyon and André Elisseeff. 2003. An Introduction to Variable and Feature Selection. Journal of Machine Learning Research 3 (2003), 1157-1182.
  15. Sara Hajian and Josep Domingo-Ferrer. 2013. A Methodology for Direct and Indirect Discrimination Pre- vention in Data Mining. IEEE Trans. Knowl. Data Eng. 25, 7 (2013), 1445-1459.
  16. Faisal Kamiran and Toon Calders. 2009. Classification without Discrimination. In Proc. nd IC4 conf. on Computer, Control and Communication. 1-6.
  17. Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. 2010. Discrimination Aware Decision Tree Learn- ing. In Proc. of the 2010 IEEE International Conference on Data Mining (ICDM). 869-874.
  18. Faisal Kamiran, Indre Zliobaite, and Toon Calders. 2013a. Quantifying explainable discrimination and re- moving illegal discrimination in automated decision making. Knowl. Inf. Syst. 35, 3 (2013), 613-644.
  19. Faisal Kamiran, Indre Zliobaite, and Toon Calders. 2013b. Quantifying explainable discrimination and re- moving illegal discrimination in automated decision making. Knowl. Inf. Syst. 35, 3 (2013), 613-644.
  20. Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012. Fairness-Aware Classifier with Prejudice Remover Regularizer. In Proc. of European Conference on Machine Learning and Knowledge Discovery in Databases (ECMLPKDD). 35-50.
  21. Binh Thanh Luong, Salvatore Ruggieri, and Franco Turini. 2011. k-NN As an Implementation of Situation Testing for Discrimination Discovery and Prevention. In Proc. of the 17th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD). 502-510.
  22. Koray Mancuhan and Chris Clifton. 2014. Combating Discrimination Using Bayesian Networks. Artif. Intell. Law 22, 2 (2014), 211-238.
  23. Sergio Mascetti, Annarita Ricci, and Salvatore Ruggieri (Eds.). 2014. Special issue: Computational Methods for Enforcing Privacy and Fairness in the Knowledge Society. Vol. 22. Artificial Intelligence and Law. Issue 2.
  24. Dino Pedreschi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-aware data mining. In Proc. of the 14th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD). 560-568.
  25. Dino Pedreschi, Salvatore Ruggieri, and Franco Turini. 2009. Measuring Discrimination in Socially- Sensitive Decision Records. In Proc. of the SIAM Int. Conf. on Data Mining (SDM). 581-592.
  26. Dino Pedreschi, Salvatore Ruggieri, and Franco Turini. 2012. A Study of Top-k Measures for Discrimination Discovery. In Proc. of the 27th Annual ACM Symposium on Applied Computing (SAC). 126-131.
  27. Andrea Romei and Salvatore Ruggieri. 2014. A multidisciplinary survey on discrimination analysis. Knowl- edge Eng. Review 29, 5 (2014), 582-638.
  28. Andrea Romei, Salvatore Ruggieri, and Franco Turini. 2012. Discovering Gender Discrimination in Project Funding. In Proc. of the 2012 IEEE 12th Int. Conf. on Data Mining Workshops (ICDMW). 394-401.
  29. Paul R. Rosenbaum and Donald B. Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 1 (1983), 41-55. Issue
  30. Salvatore Ruggieri, Dino Pedreschi, and Franco Turini. 2010. Data Mining for Discrimination Discovery. ACM Trans. Knowl. Discov. Data 4, 2, Article 9 (May 2010), 40 pages.
  31. David Tax. 2001. One-class classification. Ph.D. Dissertation. Delft University of Technology.
  32. Grigorios Tsoumakas and Ioannis Katakis. 2007. Multi-label classification: an overview. International Jour- nal of Data Warehousing & Mining 3, 3 (2007), 1-13.
  33. John Yinger. 1986. Measuring Racial Discrimination with Fair Housing Audits: Caught in the Act. The American Economic Review 76, 5 (1986), 881-893.
  34. Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. 2013. Learning Fair Repre- sentations. In Proc. of the 30th Int. Conf. on Machine Learning. 325-333.
  35. Indre Zliobaite. 2015. On the relation between accuracy and fairness in binary classification. In The 2nd workshop on Fairness, Accountability, and Transparency in Machine Learning (FATML) at ICML'15.

Cited by

Sorted by citations

See all 16

Metadata

Authors
Zliobaite, I.
Venue
arXiv
Publisher
arXiv
Published
2015
DOI
10.48550/arxiv.1511.00148
arXiv
1511.00148
References
0
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A survey on measuring indirect discrimination in machine learning. https://app.cochat.ai/papers/10.48550/arxiv.1511.00148
The paper
Zliobaite, I. (2015). A survey on measuring indirect discrimination in machine learning. arXiv. https://doi.org/10.48550/arxiv.1511.00148