Discrimination Aware Decision Tree Learning
Open accessAbstract
Recently, the following discrimination aware classification problem was introduced: given a labeled dataset and an attribute B, find a classifier with high predictive accuracy that at the same time does not discriminate on the basis of the given attribute B. This problem is motivated by the fact that often available historic data is biased due to discrimination, e.g., when B denotes ethnicity. Using the standard learners on this data may lead to wrongfully biased classifiers, even if the attribute B is removed from training data. Existing solutions for this problem consist in “cleaning away” the discrimination from the dataset before a classifier is learned. In this paper we study an alternative approach in which the non-discrimination constraint is pushed deeply into a decision tree learner by changing its splitting criterion and pruning strategy. Experimental evaluation shows that the proposed approach advances the state-of-the-art in the sense that the learned decision trees have a lower discrimination than models provided by previous methods, with little loss in accuracy.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,812 · see citing works
(2000) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(00)90183-4 · Cited by 1,262 · see citing works
Giorgio Ausiello, A., Marchetti-Spaccamela, A., Crescenzi, P., Gambosi, G., Protasi, M., & Kann, V. (1999) · Springer eBooks · doi:10.1007/978-3-642-58412-1 · Cited by 985 · see citing works
Three naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · see citing works
Self-citationDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · see citing works
Building Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468 · see citing works
Self-citationClassifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409 · see citing works
Self-citationMeasuring Discrimination in Socially-Sensitive Decision Records
Pedreschi, D., Ruggieri, S., & Turini, F. (2009) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972795.50 · Cited by 175 · see citing works
Nearest Neighbour Classification with Monotonicity Constraints
Duivesteijn, W., & Feelders, A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87479-9_38 · Cited by 90 · see citing works
Classification with No Discrimination by Preferential Sampling
Kamiran, F., & Calders, T. T. (2010) · TU/e Research Portal · Cited by 78 · see citing works
Self-citationMining optimal decision trees from itemset lattices
Nijssen, S., & Fromont, É. (2007) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281250 · Cited by 70 · see citing works
Statistical Model for Rough Set Approach to Multicriteria Classification
Dembczyński, K., Greco, S., Kotłowski, W., & Słowiński, R. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74976-9_18 · Cited by 32 · see citing works
Learning decision trees for loss minimization in multi-class problems
Margineantu, D. D., & Dietterich, T. G. (1999) · Cited by 18 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043 · see citing works
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709 · see citing works
Fairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694 · see citing works
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687 · see citing works
A comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601 · see citing works
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598 · see citing works
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Veale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · see citing works
“There are many points for intervention in a decision support system outside of the modelling process—for example, in the training data (many systems attempting to make fairer machine learning system intervene at this point [45, 23]) or after the model has been generated [46], such as the stage between model output and…”
BackgroundFA*IR: A Fair Top-k Ranking Algorithm
Zehlike, M., Bonchi, F., Castillo, C., Hajian, S., Megahed, M., & Baeza-Yates, R. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132938 · Cited by 496 · see citing works
Fairness testing: testing software for discrimination
Galhotra, S., Brun, Y., & Meliou, A. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106277 · Cited by 405 · see citing works
FairGAN: Fairness-aware Generative Adversarial Networks
Xu, D., Yuan, S., Zhang, L., & Wu, X. (2018) · 2018 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2018.8622525 · Cited by 379 · see citing works
A Methodology for Direct and Indirect Discrimination Prevention in Data Mining
Hajian, S., & Domingo-Ferrer, J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.72 · Cited by 312 · see citing works
A multidisciplinary survey on discrimination analysis
Romei, A., & Ruggieri, S. (2014) · The Knowledge Engineering Review · doi:10.1017/s0269888913000039 · Cited by 288 · see citing works
Decision Theory for Discrimination-Aware Classification
Kamiran, F., Karim, A., & Zhang, X. (2012) · 2012 IEEE 12th International Conference on Data Mining · doi:10.1109/icdm.2012.45 · Cited by 300 · see citing works
Self-citationMeasuring discrimination in algorithmic decision making
Žliobaitė, I. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-017-0506-1 · Cited by 272 · see citing works
Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification
d'Alessandro, B., O'Neil, C., & LaGatta, T. (2017) · Big Data · doi:10.1089/big.2016.0048 · Cited by 195 · see citing works
Why is my classifier discriminatory?
Chen, I., Johansson, F. D., & Sontag, D. (2018) · arXiv · doi:10.48550/arxiv.1805.12002 · Cited by 185 · see citing works
“, 2011), constraints (Kamiran et al., 2010; Zafar et al., 2017), and representation learning (Zemel et al.”
BackgroundMethodFairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramèr, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2015) · arXiv · doi:10.48550/arxiv.1510.02377 · Cited by 171 · see citing works
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramer, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2017) · 2017 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2017.29 · Cited by 171 · see citing works
Investigating the Impact of Gender on Rank in Resume Search Engines
Chen, L., Ma, R., Hannák, A., & Wilson, C. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174225 · Cited by 155 · see citing works
Handling Conditional Discrimination
liobaite, I., Kamiran, F., & Calders, T. (2011) · 2011 IEEE 11th International Conference on Data Mining · doi:10.1109/icdm.2011.72 · Cited by 125 · see citing works
Self-citationFairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Why Unbiased Computational Processes Can Lead to Discriminative Decision Procedures
Calders, T., & Žliobaitė, I. (2013) · Studies in Applied Philosophy, Epistemology and Rational Ethics · doi:10.1007/978-3-642-30487-3_3 · Cited by 128 · see citing works
Self-citationAdaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification
Krasanakis, E., Spyromitros-Xioufis, E., Papadopoulos, S., & Kompatsiaris, Y. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186133 · Cited by 127 · see citing works
Žliobaitė, I., & Custers, B. (2016) · Artificial Intelligence and Law · doi:10.1007/s10506-016-9182-5 · Cited by 125 · see citing works
Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making
Aghaei, S., Azizi, M. J., & Vayanos, P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011418 · Cited by 135 · see citing works
Kamiran, F., Žliobaitė, I., & Calders, T. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0584-8 · Cited by 99 · see citing works
Self-citationDiscrimination- and privacy-aware patterns
Hajian, S., Domingo-Ferrer, J., Monreale, A., Pedreschi, D., & Giannotti, F. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0393-7 · Cited by 93 · see citing works
Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists
Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., & Wilson, C. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317595 · Cited by 92 · see citing works
FAHT: An Adaptive Fairness-aware Decision Tree Classifier
Zhang, W., & Ntoutsi, E. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/205 · Cited by 85 · see citing works
AdaFair: Cumulative Fairness Adaptive Boosting
Iosifidis, V., & Ntoutsi, E. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357974 · Cited by 75 · see citing works
The Discriminating (Pricing) Actuary
Frees, E. W. (., & Huang, F. (2023) · North American Actuarial Journal · doi:10.1080/10920277.2021.1951296 · Cited by 56 · see citing works
“, Thouvenin et al. (2019). Like other major industries, insurers have at their disposal increasing amounts of information available about current and potential customers.”
InfluentialA survey on datasets for fairness‐aware machine learning
Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E. (2022) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1452 · Cited by 261 · see citing works
Anti-discrimination learning: a causal modeling-based framework
Zhang, L., & Wu, X. (2017) · International Journal of Data Science and Analytics · doi:10.1007/s41060-017-0058-x · Cited by 48 · see citing works
Berendt, B., & Preibusch, S. (2014) · Artificial Intelligence and Law · doi:10.1007/s10506-013-9152-0 · Cited by 44 · see citing works
The Fairness of Credit Scoring Models
Hurlin, C., Perignon, C., & Saurin, S. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3785882 · Cited by 26 · see citing works
Exploring Discrimination: A User-centric Evaluation of Discrimination-Aware Data Mining
Berendt, B., & Preibusch, S. (2012) · 2012 IEEE 12th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2012.109 · Cited by 24 · see citing works
Local data debiasing for fairness based on generative adversarial training.
Aïvodji, U., Bidet, F., Gambs, S., Ngueveu, R. C., & Tapp, A. (2021) · Algorithms · doi:10.3390/a14030087 · Cited by 16 · see citing works
- Authors
- Kamiran, F., Calders, T., & Pechenizkiy, M.
- Affiliations
- Eindhoven University of Technology (NL)
- Venue
- 2010 IEEE International Conference on Data Mining
- Publisher
- IEEE
- Published
- 2010-12
- Pages
- 869-874
- ISSN
- 1550-4786 (ISSN-L)
- OpenAlex
- W2150997454
- Cited by
- 303
- References
- 14
- Citations by year
- 2026: 1 · 2025: 24 · 2024: 28 · 2023: 29 · 2022: 35 · 2021: 33 · 2020: 26 · 2019: 31 · 2018: 15 · 2017: 21 · 2016: 10 · 2015: 8 · 2014: 17 · 2013: 7 · 2012: 16
- FWCI
- 44.26
- Type
- conference paper
- Language
- en
- Open access
- green
- research.tue.nl
- Licence
- other-oa
- Locations
- Proceedings (publishedVersion)
- TU/e Research Portal (publishedVersion)
- TU/e Research Portal (submittedVersion)
- TU/e Research Portal (submittedVersion, other-oa)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- decision tree, algorithmic fairness, training data bias, splitting criteria, tree pruning, ethnic prejudice, prediction accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
