The DOI matches its Crossref record; 10 of the 11 references it cites were found in a registry or an index.
Building Classifiers with Independency Constraints
Abstract
In this paper we study the problem of classifier learning where the input data contains unjustified dependencies between some data attributes and the class label. Such cases arise for example when the training data is collected from different sources with different labeling criteria or when the data is generated by a biased decision process. When a classifier is trained directly on such data, these undesirable dependencies will carry over to the classifier's predictions. In order to tackle this problem, we study the classification with independency constraints problem: find an accurate model for which the predictions are independent from a given binary attribute. We propose two solutions for this problem and present an empirical validation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Prior work on detecting discrimination has focused on measuring correlation or mutual information between inputs and outputs [79], discrepancies in the fractions of inputs that produce a given output [18, 19, 39-42, 87-89, 91], or discrepancies in output probability distributions [51].”
From Fairness testing: testing software for discrimination (2017) · cites this paper as [18]
In the citing paper
“Reducing discrimination (CV score) in classifiers [18, 40, 88, 91], as our evaluation has shown, often fails to remove causal discrimination and discrimination against certain groups.”
From Fairness testing: testing software for discrimination (2017) · cites this paper as [18]
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,945
Domingos, P. (1999) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/312129.312220 · Cited by 1,325
Discrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676
Chan, P. K., & Stolfo, S. J. (1998) · Cited by 452
Classifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409
Self-citationMeasuring Discrimination in Socially-Sensitive Decision Records
Pedreschi, D., Ruggieri, S., & Turini, F. (2009) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972795.50 · Cited by 175
Nearest Neighbour Classification with Monotonicity Constraints
Duivesteijn, W., & Feelders, A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87479-9_38 · Cited by 90
Statistical Model for Rough Set Approach to Multicriteria Classification
Dembczyński, K., Greco, S., Kotłowski, W., & Słowiński, R. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74976-9_18 · Cited by 32
Learning decision trees for loss minimization in multi-class problems
Margineantu, D. D., & Dietterich, T. G. (1999) · Cited by 18
Cited by
Sorted by citations
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567
“…may be possible to direct algorithms not to consider sensitive attributes that contribute to discrimination (Barocas and Selbst, 2015), such as gender or ethnicity (Calders et al., 2009; Kamiran and Calders, 2010; Schermer, 2011), based upon the emergence of discrimination in a particular context.”
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922
“Finally the idea behind the third preprocessing technique, as initially presented in [75], is to apply different weights to the instances of the dataset based on frequency counts with respect to the sensitive column.”
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
“[2] have also previously examined one method for changing the data in which different data items are given weights and the weights are adjusted to achieve fairness.”
BackgroundMethodData preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305
“We further explore the problem of discrimination in [4] where we concentrate on the case where a labeled dataset is given, and one binary attribute B which we do not want the predictions to correlate with.”
BackgroundInfluentialSelf-citationBias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043
Three naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764
“In a series of recent papers (Pedreschi et al. 2008, 2009; Kamiran and Calders 2009; Calders et al. 2009), the topic of discrimination in data mining received quite some attention.”
BackgroundSelf-citationAlgorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750
“This is because there can be other input features that may be unknown confounders that correlate with membership to a protected group 58–62 .”
BackgroundGender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684
“Various other approaches have been proposed to produce “fair” classifiers (Calders et al., 2009; Feldman et al., 2015; Misra et al., 2016).”
BackgroundFairness of Exposure in Rankings
Singh, A., & Joachims, T. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220088 · Cited by 673
“aint in the form fT Pg = h which one can plug it into the linear program from Section §3.3. We call this a Demographic Parity Constraint similar to an analogous constraint in fair supervised learning [6, 38]. Similar to that setting, in our case also, such a constraint may lead to a big loss in utility in cases when t…”
BackgroundInfluentialExplainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., van Gerven, M. A. J., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 551
Fairness testing: testing software for discrimination
Galhotra, S., Brun, Y., & Meliou, A. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106277 · Cited by 405
“, [18, 40, 88, 91]) produce fair algorithms? RQ2: How effective do the optimizations from Section 4 make Themis at identifying discrimination in software?”
BackgroundMethodResultInfluentialFairGAN: Fairness-aware Generative Adversarial Networks
Xu, D., Yuan, S., Zhang, L., & Wu, X. (2018) · 2018 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2018.8622525 · Cited by 379
Discrimination Aware Decision Tree Learning
Kamiran, F., Calders, T., & Pechenizkiy, M. (2010) · 2010 IEEE International Conference on Data Mining · doi:10.1109/icdm.2010.50 · Cited by 303
“From an ethical and legal point of view it is of course unacceptable that a model discriminating in this way is deployed; instead, it is preferable that the class assigned by the model is independent of this discriminatory attribute.”
Self-citationOn the apparent conflict between individual and group fairness
Binns, R. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372864 · Cited by 312
“gender, race) in each outcome class, and hence are classed as group fairness measures [10, 39].”
BackgroundOneto, L., & Chiappa, S. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-43883-8_7 · Cited by 328
“Pre-Processing methods consist in transforming the training data to remove undesired biases [16, 21, 24, 28, 47, 51, 52, 53, 61, 64, 65, 66, 67, 86, 90, 91, 92, 112, 114, 116, 118, 121, 122, 127, 126, 164, 174, 188, 190, 191, 193].”
BackgroundMethodYang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375709 · Cited by 254
“Researchers have proposed multiple fairness metrics including statistical parity [12, 13, 24, 39], disparate impact [27, 80], equalized odds [34] and individual fairness [23], and analyzed the relationship between them [43, 60].”
BackgroundUser-oriented Fairness in Recommendation
Li, Y., Chen, H., Fu, Z., Ge, Y., & Zhang, Y. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449866 · Cited by 225
“Demographic parity, also called independence or statistical parity, is one of the most well-known criteria for fairness [10].”
BackgroundAequitas: A Bias and Fairness Audit Toolkit
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018) · arXiv · doi:10.48550/arxiv.1811.05577 · Cited by 496
Fairness in Decision-Making — The Causal Explanation Formula
Zhang, J., & Bareinboim, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11564 · Cited by 219
Fairness in credit scoring: Assessment, implementation and profit implications
Kozodoi, N., Jacob, J., & Lessmann, S. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.06.023 · Cited by 198
“As a post-processing method, reject option classification is based on the output of a learned classifier (Kamiran et al., 2012).”
MethodTrustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
“Tabular Data Classification [60, 61, 153, 167, 194, 251] Regression [21, 38] Clustering [31, 81]”
MethodFrom individual to group privacy in big data analytics
Mittelstadt, B. (2017) · Philosophy & Technology · doi:10.1007/s13347-017-0253-7 · Cited by 184
“…algorithms from being applied to certain types of questions or purposes or (3) prevent classifier algorithms from being provided with inputs containing protected attributes (Barocas and Selbst 2015), such as gender or ethnicity (Calders et al. 2009; Kamiran and Calders 2010; Schermer 2011).”
BackgroundAn empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
“This form captures demographic parity (Ŷ ⊥ A or fθ ⊥ A) [21, 35, 65], equalized odds (Ŷ ⊥ A | Y or fθ ⊥ A | Y ) [22], and equal opportunity (Ŷ ⊥ A | Y = 1 or fθ ⊥ A | Y = 1) [22], among others [66].”
BackgroundAlgorithmic fairness in computational medicine
Xu, J., Xiao, Y., Wang, W. H., Ning, Y., Shenkman, E. A., Bian, J., & Wang, F. (2022) · eBioMedicine · doi:10.1016/j.ebiom.2022.104250 · Cited by 170
iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making
Lahoti, P., Gummadi, K. P., & Weikum, G. (2019) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2019.00121 · Cited by 175
An adversarial training framework for mitigating algorithmic biases in clinical machine learning
Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y., & Clifton, D. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00805-y · Cited by 159
Empirical Risk Minimization under Fairness ConstraintsPreprint
Donini, M., Oneto, L., Ben-David, S., Shawe-Taylor, J. S., & Pontil, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08626 · Cited by 190
“Several notions of fairness and associated learning methods have been introduced in machine learning in the past few years, including Demographic Parity [14], Equal Odds and Equal Opportunities [2], Disparate Treatment, Impact, and mistreatment [3].”
BackgroundInvestigating the Impact of Gender on Rank in Resume Search Engines
Chen, L., Ma, R., Hannák, A., & Wilson, C. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174225 · Cited by 155
“Thus, most existing fair classification systems optimize for individual [50, 53, 66, 52, 20] or group fairness [48, 9, 49, 11, 96, 51, 10, 34, 24].”
BackgroundInterventional Fairness: Causal Database Repair for Algorithmic Fairness
Salimi, B., Rodriguez, L., Howe, B., & Suciu, D. (2019) · Proceedings of the 2019 International Conference on Management of Data · doi:10.1145/3299869.3319901 · Cited by 145
“Fairness Metric Description Demographic Parity (DP) [5, 13, 47] S⊥O Conditional Statistical parity [10] S⊥O |A Equalized Odds (EO) [19, 57] S⊥O |Y Predictive Parity (PP)[9, 9, 19, 47] S⊥ Y |O Figure 2: Common associational definitions of fairness.”
BackgroundPreventing undesirable behavior of intelligent machines
Thomas, P. S., da Silva, B. C., Barto, A. G., Giguere, S., Brun, Y., & Brunskill, E. (2019) · Science · doi:10.1126/science.aag3311 · Cited by 156
“Balancing the training data and introducing noise by flipping the labels of some feature vectors and repeating some training data can improve demographic parity for näıve Bayes classifiers as well [125].”
BackgroundTowards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Brun, Y., & Meliou, A. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264838 · Cited by 138
“Group discrimination is a generalization of the Calders-Verwer (CV) score [17], used frequently in prior work on algorithmic fairness, particularly in the context of fair machine learning [16, 32, 73, 75].”
MethodHandling Conditional Discrimination
liobaite, I., Kamiran, F., & Calders, T. (2011) · 2011 IEEE 11th International Conference on Data Mining · doi:10.1109/icdm.2011.72 · Cited by 125
“The previous works solve the problem by introducing a reverse discrimination either in the training data [2], [5] or pushing constraints into the trained classifiers [3], [4].”
BackgroundMethodSelf-citationAdaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification
Krasanakis, E., Spyromitros-Xioufis, E., Papadopoulos, S., & Kompatsiaris, Y. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186133 · Cited by 127
“Such approaches include massaging the dataset [3, 13, 16, 24–26] by changing class labels that are identified as mislabeled due to bias and reweighting (usually heuristically) training samples so that more importance is placed on sensitive ones [3, 11, 25].”
BackgroundCrowds, Lending, Machine, and Bias
Fu, R., Huang, Y. H., & Singh, P. V. (2021) · Information Systems Research · doi:10.1287/isre.2020.0990 · Cited by 132
EDITS: Modeling and Mitigating Data Bias for Graph Neural Networks
Dong, Y., Liu, N., Jalaian, B. A., & Li, J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512173 · Cited by 125
“There are already debiasing approaches modifying original datasets via perturbing data distributions or reweighting the data points in the dataset [8, 25, 57].”
A scoping review of artificial intelligence-based methods for diabetes risk prediction
Mohsen, F., Al-Absi, H. R. H., Yousri, N. A., El Hajj, N., & Shah, Z. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00933-5 · Cited by 125
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124
Fair Attribute Classification through Latent Space De-biasing
Ramaswamy, V., Kim, S. S. Y., & Russakovsky, O. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00918 · Cited by 122
“Multiple metrics have been proposed to measure fairness of a model [19, 44, 46, 10, 11] and each of these measures a different notion of fairness.”
BackgroundA Survey on Trustworthy Recommender Systems
Ge, Y., Liu, S., Fu, Z., Tan, J., Li, Z., Xu, S., Li, Y., Xian, Y., & Zhang, Y. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3652891 · Cited by 171
“Based on the above definitions, many variants are derived, such as Equal Opportunity, which requires that the true positive rate is the same across different groups [122], Equalized Odds, which requires that different groups should have the same true positive rate and false positive rate [122, 325], as well as Demogra…”
BackgroundComparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression
Park, Y., Hu, J., Singh, M., Sylla, I., Dankwa‐Mullan, I., Koski, E., & Das, A. K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.3909 · Cited by 116
Sustainable MLOps: Trends and Challenges
Tamburri, D. A. (2020) · 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc51798.2020.00015 · Cited by 124
“, the extent to which a software gives fair allocation of decision power to a group of arbiters or stakeholders [26]—which typically individually map to a specific domain.”
BackgroundAlgorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 161
“Their status as de facto fairness benchmarks is probably due to their use in seminal works (Pedreshi et al. 2008; Calders et al. 2009) and influential articles (Angwin et al. 2016) on algorithmic fairness.”
BackgroundWeakly-Supervised Disentanglement Without CompromisesPreprint
Locatello, F., Poole, B., Rätsch, G., Schölkopf, B., Bachem, O., & Tschannen, M. (2020) · International Conference on Machine Learning · doi:10.48550/arxiv.2002.02886 · Cited by 382
“While they observed a strong correlation between demographic parity (Calders et al., 2009; Zliobaite, 2015) and disentanglement, the applicability of their approach is limited by the fact that disentangled representations are difficult to identify without access to explicit observations of the…”
BackgroundInfluentialA Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifiers
Chen, Z., Zhang, J. M., Sarro, F., & Harman, M. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3583561 · Cited by 130
Policy Learning for Fairness in RankingPreprint
Singh, A., & Joachims, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04056 · Cited by 122
“In the work that does exist, some consider group fairness in rankings along the lines of demographic parity (Zliobaite, 2015; Calders et al., 2009), proposing definitions and methods that minimize the difference in the representation between groups in a prefix of the ranking (Yang and Stoyanovich, 2017; Celis et al.”
BackgroundDoes Object Recognition Work for Everyone?Preprint
DeVries, T., Misra, I., Wang, C., & van der Maaten, L. J. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02659 · Cited by 141
“For instance, statistical parity [5, 6, 14, 23–25, 30, 49] states that in binary-classification settings, members of different groups should have the same chance of receiving a positive prediction.”
BackgroundKamiran, F., Žliobaitė, I., & Calders, T. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0584-8 · Cited by 99
“Next we investigate to what extent the two existing global techniques [7, 24] remove illegal discrimination.”
BackgroundMethodInfluentialSelf-citationThemis: automatically testing software for discrimination
Angell, R., Johnson, B., Brun, Y., & Meliou, A. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264590 · Cited by 94
Retiring Adult: New Datasets for Fair Machine LearningPreprint
Ding, F., Hardt, M., Miller, J. A., & Schmidt, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04884 · Cited by 137
Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists
Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., & Wilson, C. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317595 · Cited by 92
“There is a large and growing literature on how to achieve these objectives in machine learning-based classifiers [5–7, 14, 17, 22, 28–33, 39, 55].”
BackgroundCounterfactual risk assessments, evaluation, and fairness
Coston, A., Mishler, A., Kennedy, E. H., & Chouldechova, A. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372851 · Cited by 107
“Base rate parity is similar to the fairness notion of demographic parity, which requires Ŷ ⊥ A [5, 15, 54].”
BackgroundOn the Privacy Risks of Algorithmic Fairness
Chang, H., & Shokri, R. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00028 · Cited by 107
Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality
Khademi, A., Lee, S., Foley, D., & Honavar, V. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313559 · Cited by 95
“Some examples include: fairness through unawareness [14], individual fairness [11], equalized odds [15, 54], calibration [8], demographic (or statistical) parity [6, 21–23], the 80% rule (disparate impact) [13, 55], representational fairness [33, 56], and fairness under composition [12].”
BackgroundOperationalizing individual fairness with pairwise fair representations
Lahoti, P., Gummadi, K. P., & Weikum, G. (2019) · Proceedings of the VLDB Endowment · doi:10.14778/3372716.3372723 · Cited by 115
“Approaches to achieve group fairness include debiasing the input data via data perturbation, re-sampling, modifying the value of protected attribute/class labels [33, 23, 32, 16] as well as incorporating demographic parity as an additional constraint in the objective function of machine learning models [25, 8, 38].”
BackgroundHow Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions
Cheng, H.-F., Stapleton, L., Kawakami, A., Sivaraman, V., Cheng, Y., Qing, D., Perer, A., Holstein, K., Wu, Z. S., & Zhu, H. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501831 · Cited by 107
Saghiri, A. M., Vahidipour, S. M., Jabbarpour, M. R., Sookhak, M., & Forestiero, A. (2022) · Applied Sciences · doi:10.3390/app12084054 · Cited by 122
Accuracy and fairness trade-offs in machine learning: a stochastic multi-objective approach
Liu, S., & Vicente, L. N. (2022) · Computational Management Science · doi:10.1007/s10287-022-00425-z · Cited by 83
“From the perspective of multi-objective optimization (MOO), most of the in-training methods in the literature (Barocas & Selbst, 2016; Calders et al., 2009; Kamishima et al., 2011; Woodworth et al., 2017; Zafar et al., 2017a;b) are based on the so-called a priori methodology, where the…”
InfluentialBursting the Filter Bubble: Fairness-Aware Network Link Prediction
Masrour, F., Wilson, T., Yan, H., Tan, P., & Esfahanian, A. H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5429 · Cited by 86
“…for assessing fairness is demographic parity (Louizos et al. 2015; Kamishima, Akaho, and Sakuma 2011; Johndrow, Lum, and others 2019; Edwards and Storkey 2015; Calders, Kamiran, and Pechenizkiy 2009), which considers the degree of independence between the model output and protected attribute.”
BackgroundFAHT: An Adaptive Fairness-aware Decision Tree Classifier
Zhang, W., & Ntoutsi, E. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/205 · Cited by 85
Fairea: a model behaviour mutation approach to benchmarking bias mitigation methods
Hort, M., Zhang, J. M., Sarro, F., & Harman, M. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468565 · Cited by 90
“2.1 Udeshi et al. [58] ASE Automated approach to discover inputs that highlight fairness violations. accuracy changes in separate graphs, or visualise them in a 2-dimensional graph (one dimension is accuracy, the other dimen-sion is fairness) [12, 14–16, 24, 36, 37, 39, 41, 42, 51, 60].”
BackgroundMethodInfluential"Ignorance and Prejudice" in Software Fairness
Zhang, J. M., & Harman, M. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00129 · Cited by 88
“[44] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy.”
Background'Un'Fair Machine Learning Algorithms
Fu, R., Aseri, M., Singh, P. V., & Srinivasan, K. (2019) · Management Science · doi:10.1287/mnsc.2021.4065 · Cited by 85
Classification with No Discrimination by Preferential Sampling
Kamiran, F., & Calders, T. T. (2010) · TU/e Research Portal · Cited by 78
Self-citationInherent Tradeoffs in Learning Fair RepresentationsPreprint
Zhao, H., & Gordon, G. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08386 · Cited by 87
“On the other hand, while it has long been observed that demographic parity may eliminate perfect predictor [14], and previous work has empirically verified that tradeoff exists between accuracy and demographic parity [6, 17, 38] on various datasets, so far a quantitative characterization on the exact tradeoff between…”
ResultInfluentialAchieving Causal Fairness through Generative Adversarial Networks
Xu, D., Wu, Y., Yuan, S., Zhang, L., & Wu, X. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/201 · Cited by 76
Active Fairness in Algorithmic Decision Making
Noriega-Campero, A., Bakker, M. A., Bueno, B. G. B., & Pentland, A. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314277 · Cited by 74
AdaFair: Cumulative Fairness Adaptive Boosting
Iosifidis, V., & Ntoutsi, E. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357974 · Cited by 75
“re-weighting [4], uniform- or preferential-sampling [20] and data augmentation [17] are pre-processing methods which change data distributions directly; they aim to restore balance among protected and non-protected group within data by changing instance labels, assigning different weights, under- or over-sampling inst…”
MethodPerrone, V., Donini, M., Zafar, M. B., Schmucker, R., Kenthapadi, K., & Archambeau, C. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462629 · Cited by 66
Towards Equity and Algorithmic Fairness in Student Grade Prediction
Jiang, W., & Pardos, Z. A. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462623 · Cited by 70
“race), the overall probability of a positive prediction of a given outcome should be the same - the sensitive attribute should be independent of the prediction [10], i.”
Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions
Ho, J. Q. H., Hartanto, A., Koh, A., & Majeed, N. M. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100145 · Cited by 83
Establishing and evaluating trustworthy AI: overview and research challenges
Kowald, D., Scher, S., Pammer-Schindler, V., Müllner, P., Waxnegger, K., Demelius, L., Fessl, A., Toller, M., Mendoza Estrada, I. G., Šimić, I., et al. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1467222 · Cited by 59
Probabilistic verification of fairness properties via concentration
Bastani, O., Zhang, X., & Solar-Lezama, A. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360544 · Cited by 57
“Intuitively, our first property says that minority members should be classified as f (V ) = 1 at approximately the same rate as majority candidates [Calders et al. 2009].”
BackgroundMethodInfluentialOn the Fairness of Disentangled RepresentationsPreprint
Locatello, F., Abbati, G., Rainforth, T., Bauer, S., Schölkopf, B., & Bachem, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13662 · Cited by 78
“To evaluate the learned representations r(x) of these observations, we assume that the set of ground-truth factors of variation include both a target factor y, which we would like to predict from the learned representation, and an underlying sensitive factor s, which we want to be fair to in the sense of demographic p…”
BackgroundFairness in Semi-Supervised Learning: Unlabeled Data Help to Reduce Discrimination
Zhang, T., Zhu, T., Li, J., Han, M., Zhou, W., & Yu, P. S. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3002567 · Cited by 60
“edu} Over the past few years, much research has been devoted to designing fairness metrics, such as statistical fairness [1], [3], [4], [5], individual fairness [6], [7], [8] and causal fairness [9], [10].”
InfluentialA survey on datasets for fairness‐aware machine learning
Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E. (2022) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1452 · Cited by 261
Ryan, M., Antoniou, J., Brooks, L., Jiya, T., Macnish, K., & Stahl, B. C. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-021-00293-x · Cited by 65
Fair Generative Modeling via Weak SupervisionPreprint
Choi, K., Grover, A., Singh, T., Shu, R., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12008 · Cited by 58
“It has often been used in the context of fair classification (Calders et al., 2009), for example, (Kamiran & Calders, 2012) details reweighting as a way to remove discrimination without relabeling instances.”
MethodBalancing Learning Model Privacy, Fairness, and Accuracy With Early Stopping Criteria
Zhang, T., Zhu, T., Gao, K., Zhou, W., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3129592 · Cited by 50
“Definition 4 (Demographic Parity [18]): A classification model satisfies the demographic parity if”
BackgroundImproving Recommendation Fairness via Data Augmentation
Chen, L., Wu, L., Zhang, K., Hong, R., Lian, D., Zhang, Z., Zhou, J., & Wang, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583341 · Cited by 50
“According to specific tasks, researchers have proposed specific demographic parity measures [10, 35, 56].”
BackgroundOn the Moral Justification of Statistical Parity
Hertweck, C., Heitz, C., & Loi, M. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445936 · Cited by 59
“Independence has been particularly influential in the early stages of algorithmic fairness [4, 20] while the two other fairness crite-”
BackgroundFairness Violations and Mitigation under Covariate Shift
Singh, H., Singh, R., Mhasawade, V., & Chunara, R. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445865 · Cited by 61
“(DP) [7] A classifier f is said to satisfy demographic parity for some distribution P if P (f (S) |A) = P (f (S)).”
BackgroundUnderstanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning
Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., & Chi, E. H. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467326 · Cited by 50
“Although being adopted as the definition of fairness in a series of works [9, 24, 25], demographic parity may be unrealistic to achieve in practice, especially in the common scenarios where the base rates differ across subgroups [60].”
BackgroundFairness Improvement with Multiple Protected Attributes: How Far Are We?
Chen, Z., Zhang, J. M., Sarro, F., & Harman, M. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639083 · Cited by 53
“Finding 5: It is challenging for fairness improvement methods to achieve good fairness-performance trade-offs for imbalanced datasets and applications where precision matters when dealing with multiple protected attributes.”
BackgroundTutorial on Fairness of Machine Learning in Recommender Systems
Li, Y., Ge, Y., & Zhang, Y. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462814 · Cited by 46
“The group fairness perspective for supervised learning usually implies constraints such as equalized odds [32, 60] and demographic parity [11].”
BackgroundOn Dyadic Fairness: Exploring and Mitigating Bias in Graph Connections
Li, P., Wang, Y., Zhao, H., Hong, P., & Liu, H. (2021) · International Conference on Learning Representations · Cited by 44
Analyzing the impact of missing values and selection bias on fairness
Wang, Y., & Singh, L. (2021) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00259-z · Cited by 55
“Much literature has focused on algorithms for fixing bias, where the algorithms are validated on fairly “clean” data sets [12], including the COMPAS recidivism data, the German credit data, and the adult income data [20,31].”
MethodInclusive GAN: Improving Data and Minority Coverage in Generative Models
Yu, N., Li, K., Zhou, P., Malik, J., Davis, L., & Fritz, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_23 · Cited by 48
FARF: A Fair and Adaptive Random Forests Classifier
Zhang, W., Bifet, A., Zhang, X., Weiss, J. C., & Nejdl, W. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75765-6_20 · Cited by 43
“These studies have typically adopted one or more of the three following strategies: i) Pre-processing solutions [8,9,10] aim to eliminate discrimination at the data level, including the most popular ones massaging [11] and reweighting [12].”
BackgroundConditional Learning of Fair Representations
Zhao, H., Coston, A., Adel, T., & Gordon, G. J. (2019) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · doi:10.48550/arxiv.1910.07162 · Cited by 75
“However, it has long been empirically observed (Calders et al., 2009) and recently been proved (Zhao & Gordon, 2019) that fairness is often at odds with utility.”
BackgroundMethodFairMask: Better Fairness via Model-Based Rebalancing of Protected Attributes
Peng, K., Chakraborty, J., & Menzies, T. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3220713 · Cited by 42
“Moreover, many prior works in the literature attempt to mitigate bias by modifying the protected attributes in their methods [8], [31], [45], [61], [62], [63].”
BackgroundAssessing trustworthy AI: Technical and legal perspectives of fairness in AI
Kattnig, M., Angerschmid, A., Reichel, T., & Kern, R. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.106053 · Cited by 63
Fairness for Robust Log Loss Classification
Rezaei, A., Fathony, R., Memarrast, O., & Ziebart, B. D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6002 · Cited by 44
“Though our approach can be applied to a wider range of fairness constraints, we focus on three prominent ones: Demographic Parity [12], Equality of Opportunity [13] and Equality of Odds [13].”
BackgroundMethodInfluentialEvaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information
Awasthi, P., Beutel, A., Kleindeßner, M., Morgenstern, J., & Wang, X. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445884 · Cited by 41
“For popular fairness metrics such as equalized odds or demographic parity [6, 17, 30, 32], the sensitive attribute information obtained from the classifier can then be used in conjunction with the predicted label information to get an estimate of the bias of a given model.”
MethodRobust Fairness Under Covariate Shift
Rezaei, A., Liu, A., Memarrast, O., & Ziebart, B. D. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i11.17135 · Cited by 44
Survey on fairness notions and related tensions
Alves, G. T., Bernier, F., Couceiro, M., Makhlouf, K., Catuscia Palamidessi, A., & Zhioua, S. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100033 · Cited by 47
Color-Theoretic Experiments to Understand Unequal Gender Classification Accuracy From Face Images
Muthukumar, V., Pedapati, T., Ratha, N., Sattigeri, P., Wu, C.-W., Kingsbury, B. E. D., Kumar, A., Thomas, S., Mojsilović, A., & Varshney, K. R. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00282 · Cited by 39
“Theoretical approaches that have been designed to solve the fairness problem range from defining new fairness metrics such as demographic parity [6, 46] and equality of odds/opportunity [18], to deciding how to actually design fair ML algorithms according to these metrics [3].”
BackgroundFairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination
Zhang, T., zhu, T., Li, J., Han, M., Zhou, W., & Yu, P. (2021) · OPUS - Open Publications of UTS Scholars (University of Technology Sydney) · Cited by 38
(2017) · doi:10.4230/lipics.forc.2020.6 · Cited by 27
On Adversarial Bias and the Robustness of Fair Machine LearningPreprint
Chang, H., Nguyen, T. D., Murakonda, S. K., Kazemi, E., & Shokri, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08669 · Cited by 67
“[8] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy.”
Metadata
- Authors
- Calders, T., Kamiran, F., & Pechenizkiy, M.
- Affiliations
- Eindhoven University of Technology (NL)
- Venue
- 2009 IEEE International Conference on Data Mining Workshops
- Publisher
- IEEE
- Published
- 2009-12
- Pages
- 13-18
- ISSN
- 2375-9259 (ISSN-L)
- OpenAlex
- W2116666691
- Cited by
- 468
- References
- 11
- Citations by year
- 2026: 16 · 2025: 45 · 2024: 64 · 2023: 51 · 2022: 65 · 2021: 79 · 2020: 60 · 2019: 48 · 2018: 17 · 2017: 5 · 2016: 2 · 2015: 2 · 2014: 1 · 2012: 5
- FWCI
- 2.58
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE ... International Conference on Data Mining workshops (publishedVersion)
- TU/e Research Portal (publishedVersion)
- TU/e Research Portal (submittedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- biased training data, classifier training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
