The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
Open accessAbstract
As more researchers have become aware of and passionate about algorithmic fairness, there has been an explosion in papers laying out new metrics, suggesting algorithms to address issues, and calling attention to issues in existing applications of machine learning. This research has greatly expanded our understanding of the concerns and challenges in deploying machine learning, but there has been much less work in seeing how the rubber meets the road. In this paper we provide a case-study on the application of fairness in machine learning research to a production classification system, and offer new insights in how to measure and address algorithmic fairness issues. We discuss open questions in implementing equality of opportunity and describe our fairness metric, conditional equality, that takes into account distributional differences. Further, we provide a new approach to improve on the fairness metric during model training and demonstrate its efficacy in improving performance for a real-world product.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This case study applies equality-of-opportunity fairness to a production binary classification system, introduces conditional equality to account for distributional differences, and proposes absolute correlation regularization for training. It reports reduced false positive rate gaps for two sensitive groups and argues the approach is more stable than adversarial training.
Key findings
- The baseline linear model had FPR Ratio > 1 for both groups: Group 1 = 5.3× and Group 2 = 2.18×, indicating adverse actions were more often incorrectly taken for these groups.
- Moving to a DNN reduced the Group 1 FPR gap from 2.62× to 1.44× and the Group 2 gap from 1.76× to 1.25×, but did not improve FPR everywhere; Group 2 and Not-Group 2 FPR increased.
- Adversarial training reduced the Group 1 FPR gap from 1.44× to 1.04×; absolute correlation regularization kept it approximately 1.05× while improving training stability.
- Applying correlation regularization for Group 1 slightly improved the Group 2 gap from 1.37× to 1.31×; adding two regularization terms improved the Group 2 gap to 1.11×, though it did not decrease Group 2 FPR.
Methods
- Production binary classifier predicts policy-breaking examples; adverse action is taken if predicted score exceeds a threshold. Ground truth is the average of K human raters’ scores y ∈ [0, 1] on a subsample; the model is trained as a regression model with squared error.
- Sensitive group membership s is not reliably observed at inference; a small number of users share demographics, used for training and offline evaluation. The paper compares a baseline linear model with a DNN (single hidden layer), adversarial training, and absolute correlation regularization.
- Fairness metric: conditional equality of opportunity, conditioning on feature A (e.g., averaged human rating) and weighting gaps by p_a; results are reported as FPR ratios, with error bars from training the model 10 times and averaging.
Limitations
- Stated: the paper assumes the product policy is ground truth and human raters provide unbiased estimates; rater bias is difficult to evaluate and needs further research. Sensitive attributes are unavailable at inference, and limited demographic data prevents expansive intersectional evaluation.
- Stated: evaluation uses examples currently in the system, whose distribution is affected by the system’s previous performance; only binary actions are considered, and continuous scores or composition with other signals are not addressed.
- Evident: this is a single production case study with two sensitive groups and relativistic FPR ratios; absolute correlation regularization is not provably optimal for the fairness metric, and numerical y-axis values are omitted due to sensitivity.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Own result · 4 Not checked yet · 2 Own method · 2 Own interpretation · 1 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that for both Group 1 and Group 2 the FPR Ratio > 1, with FPR Ratio Group1 = 5.3× and FPR Ratio Group2 = 2.18×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 3.1 Baseline Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we find these are sometimes unstable and difficult to reliably train well.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 1 INTRODUCTION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe in Figure 3d that using absolute correlation regularization keeps the FPR gap approximately the same (1.05×).”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“different groups of users are associated with different types of examples, such as with different use cases or target audiences.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 4.1 Data Distribution
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that absolute correlation regularization stably and significantly improves algorithmic fairness metrics.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 1 INTRODUCTION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we see in Figure 4d, we are able to improve the FPR gap for Group 2 to 1.11×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that different ways of generating that evaluation data give significantly different results.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 4.1 Data Distribution
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that for both Group 1 and Group 2 the FPR Ratio > 1, with FPR Ratio Group1 = 5.3× and FPR Ratio Group2 = 2.18×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 3.1 Baseline Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we observe that the FPR does increase with the averaged human rater score”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 4.2 Distribution-Dependent Metrics
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Figure 4c we observe a very slight improvement in the FPR gap, bringing it down from 1.37× to 1.31×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
- It may state two separate claims as one. 6
- Probably not a substantial claim. 3
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we observe the FPR increases for Group 2 and Not-Group 2 usersfoot_1 ; the FPR gap decreases from 1.76× to 1.25×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Model Capacity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As can be seen in Figure 3b, the move to DNN decreases the FPR for Group 1, as well as decreasing the FPR gap from 2.62× to 1.44×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Model Capacity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we see in Figure 3c, this significantly decreases the FPR gap from 1.44× to 1.04× and simultaneously decreases the FPR for Group 1, despite that not being an explicit objective.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
In the paper
“Hardt et al. [21] address this by analyzing the accuracy and asserting that the model should not mistake y = 0 examples for y = 1 examples at a higher rate for one group than another, called equality of opportunity”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 2 BACKGROUND AND RELATED WORK
What the sources say
- Not checked yetcites Hardt et al. [21]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while increasing model capacity may help in some cases, it does not necessarily improve accuracy everywhere.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Model Capacity.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Error bars are based on training the model 10 times and averaging results.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 6 IMPROVEMENTS IN PRACTICE
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both of these numbers indicate a gap in the equality of opportunity and that examples from these groups are more frequently incorrectly having adverse action taken against them.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 3.1 Baseline Model
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we use only data for which y < τ , as we are concerned with the FPR.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The authors state this as an assumption.
The product policy is taken as ground truth.
“We take the product policy as ground truth.”
The authors state this as an assumption.
The human raters provide an unbiased estimate of the ground truth score.
“we make the simplifying assumption that the human raters provide an unbiased estimate of the ground truth score.”
The authors state this as an assumption.
The labels are unbiased.
“This work, like most of the algorithmic fairness literature, assumes that the labels are unbiased.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, adversarial approaches have been reported to often lack stability which can make them hard to train reliably [30,99] and also speciically in some transfer learning scenarios [206] when, for example, the protected variable is known only for a small number of samples.”
From Fairness in machine learning: A survey (2024) · cites this paper as [30]
In the citing paper
“Most approaches in this area use notions of fairness within the adversary to apply feedback for model tuning as a form of in-processing where the adversary penalizes the model if a sensitive variable is predictable from the dependent variable (e.g.[28,30,58,91,289,299,305,314]).”
From Fairness in machine learning: A survey (2024) · cites this paper as [30]
In the citing paper
“Additional forms of regularization have been proposed to try and address these issues (e.g.[30]).”
From Fairness in machine learning: A survey (2024) · cites this paper as [30]
Citations
Cites
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Proceedings of the 29th International Conference on Machine Learning (ICML-12)Preprint
Langford, J. C., & Pineau, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.4676 · Cited by 1,687
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., & Erhan, D. (2016) · arXiv · doi:10.48550/arxiv.1608.06019 · Cited by 1,208
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
(2013) · Cited by 1,126
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Research works printed but not linked: 2
- 10.1145/2090236.2090255 · DOI · Paper page
- Zachary C Lipton , Alexandra Chouldechova , and Julian McAuley . 2017. Does mitigating ML's disparate impact require disparate treatment? arXiv preprint arXiv:1711.07076 ( 2017 ). Zachary C Lipton, Alexandra Chouldechova, and Julian McAuley. 2017. Does mitigating ML's disparate impact require disparate treatment? arXiv preprint arXiv:1711.07076 (2017). · arXiv
Cited by
Sorted by citations
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598
Fairness in Recommendation Ranking through Pairwise Comparisons
Beutel, A., Chen, J., Doshi, T. S., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Ed, H., et al. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330745 · Cited by 374
Self-citationBias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3631326 · Cited by 322
Fairness in Deep Learning: A Computational Perspective
Du, M., Yang, F., Zou, N., & Hu, X. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.3000681 · Cited by 218
A survey on popularity bias in recommender systems
Klimashevskaia, A., Jannach, D., Elahi, M., & Trattner, C. (2024) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-024-09406-0 · Cited by 223
Pessach, D., & Shmueli, E. (2023) · Springer eBooks · doi:10.1007/978-3-031-24628-9_37 · Cited by 235
Fairness in Recommendation: Foundations, Methods, and Applications
Li, Y., Chen, H., Xu, S., Ge, Y., Tan, J., Liu, S., & Zhang, Y. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3610302 · Cited by 183
Fairway: a way to build fair ML software
Chakraborty, J., Majumder, S., Yu, Z., & Menzies, T. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409697 · Cited by 167
Building Human Values into Recommender Systems: An Interdisciplinary Synthesis
Stray, J., Halevy, A. Y., Assar, P., Hadfield-Menell, D., Boutilier, C. E., Ashar, A., Bakalar, C., Beattie, L., Ekstrand, M. D., Leibowicz, C., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3632297 · Cited by 183
Connecting user and item perspectives in popularity debiasing for collaborative recommendation
Boratto, L., Fenu, G., & Marras, M. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102387 · Cited by 139
Fairness without Demographics through Adversarially Reweighted LearningPreprint
Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., Wang, X., & Ed, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13114 · Cited by 167
Self-citationZ-Inspection ® : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 147
Fairway: A Way to Build Fair ML Software
Chakraborty, J., Majumder, S., Yu, Z., & Menzies, T. (2020) · Figshare · doi:10.6084/m9.figshare.12363347 · Cited by 107
Barnes, E., & Hutson, J. (2024) · Forum for education studies. · doi:10.59400/fes.v2i2.1229 · Cited by 115
In-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99
Addressing Marketing Bias in Product Recommendations
Wan, M., Ni, J., Misra, R., & McAuley, J. (2020) · Proceedings of the 13th International Conference on Web Search and Data Mining · doi:10.1145/3336191.3371855 · Cited by 81
Czarnowska, P., Vyas, Y., & Shah, K. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00425 · Cited by 72
Towards Equity and Algorithmic Fairness in Student Grade Prediction
Jiang, W., & Pardos, Z. A. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462623 · Cited by 70
The Role of Explainable AI in the Research Field of AI Ethics
Vainio-Pekka, H., Agbese, M., Jantunen, M., Vakkuri, V., Mikkonen, T., Rousi, R. A., & Abrahamsson, P. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3599974 · Cited by 85
Vasudevan, S., & Kenthapadi, K. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412705 · Cited by 64
Silva: Interactively Assessing Machine Learning Fairness Using Causality
Yan, J. N., Gu, Z., Lin, H., & Rzeszotarski, J. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376447 · Cited by 54
Exploiting personalized calibration and metrics for fairness recommendation
da Silva, D. C., Manzato, M. G., & Dur�ão, F. A. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115112 · Cited by 54
El–Sappagh, S., Alonso, J. M., Abuhmed, T., Ali, F., & Bugarín, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10415-5 · Cited by 58
Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning
Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., & Chi, E. H. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467326 · Cited by 50
Self-citationAddressing bias in bagging and boosting regression models
Ugirumurera, J., Bensen, E. A., Severino, J., & Sanyal, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-68907-5 · Cited by 50
Zheng, Y., Wang, S., & Zhao, J. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103410 · Cited by 51
Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach
Fabris, A., Esuli, A., Moreo, A., & Sebastiani, F. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14033 · Cited by 54
Debiasing Embeddings for Reduced Gender Bias in Text Classification
Prost, F., Thain, N., & Bolukbasi, T. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3810 · Cited by 48
Fairness and Discrimination in Retrieval and Recommendation
Ekstrand, M. D., Burke, R., & Díaz, F. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331380 · Cited by 47
Fairness and discrimination in recommendation and retrieval
Ekstrand, M. D., Burke, R., & Díaz, F. (2019) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3298689.3346964 · Cited by 46
Enabling Fairness in Healthcare Through Machine Learning
Grote, T., & Keeling, G. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09658-7 · Cited by 47
FARF: A Fair and Adaptive Random Forests Classifier
Zhang, W., Bifet, A., Zhang, X., Weiss, J. C., & Nejdl, W. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75765-6_20 · Cited by 43
Modeling and Guiding the Creation of Ethical Human-AI Teams
Flathmann, C., Schelble, B. G., Zhang, R., & McNeese, N. J. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462573 · Cited by 43
Transfer of Machine Learning Fairness across DomainsPreprint
Schumann, C., Wang, X., Beutel, A., Chen, J., Qian, H., & Ed, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09688 · Cited by 41
Self-citationHopkins, A., & Booth, S. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462527 · Cited by 40
Rodolfa, K. T., Salomon, E., Haynes, L., Mendieta, I. H., Larson, J., & Ghani, R. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372863 · Cited by 50
Longitudinal Fairness with Censorship
Zhang, W., & Weiss, J. C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21484 · Cited by 37
Ethical and methodological challenges in building morally informed AI systems
Hagendorff, T., & Danks, D. (2023) · AI and Ethics · doi:10.1007/s43681-022-00188-y · Cited by 36
Machine Learning for Fraud Detection in E-Commerce: A Research Agenda
Tax, N., de Vries, K. J., de Jong, M., Dosoula, N., van den Akker, B., Smith, J., Thuong, O., & Bernardi, L. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-87839-9_2 · Cited by 39
SoK: Content Moderation for End-to-End Encryption
Scheffler, S., & Mayer, J. R. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0060 · Cited by 43
Evaluating Fairness Using Permutation Tests
DiCiccio, C. J., Vasudevan, S., Basu, K., Kenthapadi, K., & Agarwal, D. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403199 · Cited by 34
A Step Toward More Inclusive People Annotations for Fairness
Schumann, C., Ricco, S., Prabhu, U., Ferrari, V., & Pantofaru, C. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462594 · Cited by 43
Fairness and Bias in Robot Learning
Londoño, L., Hurtado, J. V., Hertz, N., Kellmeyer, P., Vöeneky, S., & Valada, A. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3403898 · Cited by 43
State-of-the-Art in Responsible, Explainable, and Fair AI for Medical Image Analysis
Amirian, S., Gao, F., Littlefield, N. G., Hill, J. H., Yates, A. J., Plate, J. F., Pantanowitz, L., Rashidi, H. H., & Tafti, A. P. (2025) · IEEE Access · doi:10.1109/access.2025.3555543 · Cited by 34
FairOD: Fairness-aware Outlier Detection
Shekhar, S., Shah, N., & Akoglu, L. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462517 · Cited by 30
Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian InferencePreprint
Ji, D., Smyth, P., & Mark Steyvers, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09851 · Cited by 29
Quantifying representativeness in randomized clinical trials using machine learning fairness metrics
Qi, M., Cahan, O., Foreman, M. A., Gruen, D. M., Das, A. K., & Bennett, K. P. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab077 · Cited by 27
Fairness by Explicability and Adversarial SHAP Learning
Hickey, J. M., Stefano, P. G. D., & Vasileiou, V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67664-3_11 · Cited by 30
Fair Decision-making Under Uncertainty
Zhang, W., & Weiss, J. C. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00100 · Cited by 29
Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production SystemsPreprint
Bakalar, C., Barreto, R. D. C., Bergman, S., Bogen, M., Chern, B., Corbett‐Davies, S., Hall, M. A., Kloumann, I. M., Lam, M. S., Candela, J. Q., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06172 · Cited by 40
Generative AI from Theory to Practice: A Case Study of Financial Advice
Lo, A. W., & Ross, J. (2024) · doi:10.21428/e4baedd9.a1f6a281 · Cited by 25
IF-City: Intelligible Fair City Planning to Measure, Explain and Mitigate Inequality
Lyu, Y., Lu, H., Lee, M. K., Schmitt, G., & Lim, B. Y. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3239909 · Cited by 24
Censored Fairness through Awareness
Zhang, W., Hernandez‐Boussard, T., & Weiss, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26708 · Cited by 25
From failure to fusion: A survey on learning from bad machine learning models
Naser, M. Z. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103122 · Cited by 24
The Role of Artificial Intelligence in Chronic Liver Diseases and Liver Transplantation
Spann, A., Strauss, A. T., Davis, S. E., & Bhat, M. (2025) · Gastroenterology · doi:10.1053/j.gastro.2025.05.012 · Cited by 21
AI in the Law: Towards Assessing Ethical Risks
Wright, S. A. (2020) · 2020 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata50022.2020.9377950 · Cited by 13
Toward a better trade-off between performance and fairness with kernel-based distribution matchingPreprint
Prost, F., Qian, H., Chen, Q., Ed, H., Chen, J., & Beutel, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11779 · Cited by 21
Self-citationMetrics and methods for a systematic comparison of fairness-aware machine learning algorithmsPreprint
Jones, G. P., Hickey, J. M., Stefano, P. G. D., Dhanjal, C., Stoddart, L. C., & Vasileiou, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03986 · Cited by 28
Principal Fairness for Human and Algorithmic Decision-Making
Imai, K., & Jiang, Z. (2023) · Statistical Science · doi:10.1214/22-sts872 · Cited by 31
In the Name of Fairness: Assessing the Bias in Clinical Record De-identification
Xiao, Y., Lim, S., Pollard, T., & Ghassemi, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593982 · Cited by 23
Individual Fairness Under Uncertainty
Zhang, W., Wang, Z., Kim, J.-Y., Cheng, C., Oommen, T., Ravikumar, P., & Weiss, J. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230621 · Cited by 21
Black, E., Naidu, R. C. A., Ghani, R., Rodolfa, K. T., Ho, D. E., & Heidari, H. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623259 · Cited by 19
Representativeness and face-ism: Gender bias in image search
Ulloa, R., Richter, A. C., Makhortykh, M., Urman, A., & Kacperski, C. (2022) · New Media & Society · doi:10.1177/14614448221100699 · Cited by 18
Travel Demand Forecasting: A Fair AI Approach
Zhang, X., Ke, Q., & Zhao, X. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3395061 · Cited by 20
Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems
Wang, X., Thain, N., Sinha, A., Prost, F., Ed, H., Chen, J., & Beutel, A. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441732 · Cited by 18
Self-citationImproving medical machine learning models with generative balancing for equity and excellence
Theodorou, B., Danek, B. P., Tummala, V., Kumar, S. D. M., Malin, B., & Sun, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01438-z · Cited by 18
FEAT: A Fairness-Enhancing and Concept-Adapting Decision Tree Classifier
Zhang, W., & Bifet, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61527-7_12 · Cited by 17
FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., & Chau, D. H. (2019) · 2019 IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast47406.2019.8986948 · Cited by 14
A fairness-focused approach to recidivism prediction: implications for accuracy, trust, and equity
Farayola, M. M., Tal, I., Saber, T., Connolly, R., & Bendechache, M. (2025) · AI & Society · doi:10.1007/s00146-025-02452-1 · Cited by 15
RUNNER: Responsible UNfair NEuron Repair for Enhancing Deep Neural Network Fairness
Li, T., Cao, Y., Zhang, J., Zhao, S., Huang, Y., Liu, A., Guo, Q., & Liu, Y. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623334 · Cited by 16
Flexible and Adaptive Fairness-aware Learning in Non-stationary Data Streams
Zhang, W., Zhang, M., Zhang, J., Liu, Z., Chen, Z., Wang, J., Raff, E., & Messina, E. (2020) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai50040.2020.00069 · Cited by 15
Understanding User Sensemaking in Machine Learning Fairness Assessment Systems
Gu, Z., Yan, J. N., & Rzeszotarski, J. M. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450092 · Cited by 15
Fairness with censorship and group constraints
Zhang, W., & Weiss, J. C. (2023) · Knowledge and Information Systems · doi:10.1007/s10115-023-01842-5 · Cited by 13
An Interactive Approach to Bias Mitigation in Machine Learning
Wang, H., Mukhopadhyay, S., Xiao, Y., & Fang, S. (2021) · 2021 IEEE 20th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) · doi:10.1109/iccicc53683.2021.9811333 · Cited by 12
Yi, F., Singh, A., & Tao, Z. (2024) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000101 · Cited by 13
Data-Centric Factors in Algorithmic Fairness
Li, N., Goel, N., & Ash, E. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534147 · Cited by 15
Bipartite Ranking Fairness Through a Model Agnostic Ordering Adjustment
Cui, S., Pan, W., Zhang, C., & Wang, F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3290949 · Cited by 13
Disentangling and Operationalizing AI Fairness at LinkedIn
Candela, J. Q., Wu, Y., Hsu, B., Jain, S., Ramos, J., Adams, J., Hallman, R., & Basu, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594075 · Cited by 14
Pham, T.-H., Yin, C., Mehta, L. S., Zhang, X., & Zhang, P. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01813-2 · Cited by 11
Doing good by fighting fraud: Ethical anti-fraud systems for mobile payments
Din, Z. A., Venugopalan, H., Lin, H., Wushensky, A., Liu, S., & King, S. T. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00100 · Cited by 10
Machine Learning practices and infrastructures
Berman, G. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604689 · Cited by 10
Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithms
Meding, K., & Hagendorff, T. (2024) · Philosophy & Technology · doi:10.1007/s13347-023-00679-8 · Cited by 13
Can We Improve Model Robustness through Secondary Attribute Counterfactuals?
Balashankar, A., Wang, X., Packer, B., Thain, N., Ed, H., & Beutel, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.386 · Cited by 11
Self-citationRuggieri, S., Álvarez, J. M., Pugnana, A., State, L., & Turini, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26798 · Cited by 11
FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently
Cong, Z., Shi, B., Li, S., Yang, J., He, Q., & Pei, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3306378 · Cited by 9
Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility
Cui, S., Pan, W., Zhang, C., & Wang, F. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467251 · Cited by 8
Trustworthy Fairness Metric Applied to AI-Based Decisions in Food-Energy-Water
Uslu, S., Kaur, D., Rivera, S. J., Arjan Durresi, A., Durrësi, M., & Babbar‐Sebens, M. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-99587-4_37 · Cited by 8
Learning fair representations for accuracy parity
Quan, T., Zhu, F., Liu, Q., & Li, F. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105819 · Cited by 10
Technical Challenges for Training Fair Neural NetworksPreprint
Cherepanova, V., Nanda, V., Goldblum, M., Dickerson, J. P., & Goldstein, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06764 · Cited by 25
On bias and fairness in deep learning-based facial analysis
Mittal, S., Majumdar, P., Mayank Vatsa, A., & Singh, R. (2023) · Handbook of statistics · doi:10.1016/bs.host.2023.01.002 · Cited by 8
Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective
Prost, F., Awasthi, P., Blumm, N., Kumthekar, A., Potter, T., Li, W., Wang, X., Ed, H., Chen, J., & Beutel, A. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462603 · Cited by 7
Self-citationResponsible automatically processable regulation
Guitton, C., Mayer, S., Tamò‐Larrieux, A., Landuyt, D. V., Fosch‐Villaronga, E., Kamara, I., & Pałka, P. (2024) · AI & Society · doi:10.1007/s00146-024-01901-7 · Cited by 7
Fairness without Sensitive Attributes via Knowledge Sharing
Ni, H., Han, L., Chen, T., Sadiq, S. W., & Demartini, G. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659014 · Cited by 6
Fairness in vulnerable attribute prediction on social media
Beiró, M. G., & Kalimeri, K. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00855-y · Cited by 6
A Data-Centric Multi-Objective Learning Framework for Responsible Recommendation Systems
Huang, X., Lian, J., Wang, H., Liao, H., Lian, D., & Xie, X. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645324 · Cited by 5
Lingam, K., & Chittineni, S. (2025) · Intelligent Decision Technologies · doi:10.1177/18724981241301281 · Cited by 5
Metadata
- Authors
- Beutel, A., Chen, J., Doshi, T., Qian, H., Woodruff, A., Luu, C., Kreitmann, P., Bischof, J., & Chi, E. H.
- Affiliations
- Google (United States) (US)
- Google, Mountain View, CA, USA
- Venue
- Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2019-01-27
- Pages
- 453-459
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W2964023221
- Cited by
- 111
- References
- 33
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 20 · 2023: 20 · 2022: 10 · 2021: 26 · 2020: 14 · 2019: 6
- FWCI
- 14.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- algorithmic fairness, equal opportunities, fairness metrics, distributional differences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
