The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
Open accessAbstract
As more researchers have become aware of and passionate about algorithmic fairness, there has been an explosion in papers laying out new metrics, suggesting algorithms to address issues, and calling attention to issues in existing applications of machine learning. This research has greatly expanded our understanding of the concerns and challenges in deploying machine learning, but there has been much less work in seeing how the rubber meets the road. In this paper we provide a case-study on the application of fairness in machine learning research to a production classification system, and offer new insights in how to measure and address algorithmic fairness issues. We discuss open questions in implementing equality of opportunity and describe our fairness metric, conditional equality, that takes into account distributional differences. Further, we provide a new approach to improve on the fairness metric during model training and demonstrate its efficacy in improving performance for a real-world product.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This case study applies equality-of-opportunity fairness to a production binary classification system, introduces conditional equality to account for distributional differences, and proposes absolute correlation regularization for training. It reports reduced false positive rate gaps for two sensitive groups and argues the approach is more stable than adversarial training.
Key findings
- The baseline linear model had FPR Ratio > 1 for both groups: Group 1 = 5.3× and Group 2 = 2.18×, indicating adverse actions were more often incorrectly taken for these groups.
- Moving to a DNN reduced the Group 1 FPR gap from 2.62× to 1.44× and the Group 2 gap from 1.76× to 1.25×, but did not improve FPR everywhere; Group 2 and Not-Group 2 FPR increased.
- Adversarial training reduced the Group 1 FPR gap from 1.44× to 1.04×; absolute correlation regularization kept it approximately 1.05× while improving training stability.
- Applying correlation regularization for Group 1 slightly improved the Group 2 gap from 1.37× to 1.31×; adding two regularization terms improved the Group 2 gap to 1.11×, though it did not decrease Group 2 FPR.
Methods
- Production binary classifier predicts policy-breaking examples; adverse action is taken if predicted score exceeds a threshold. Ground truth is the average of K human raters’ scores y ∈ [0, 1] on a subsample; the model is trained as a regression model with squared error.
- Sensitive group membership s is not reliably observed at inference; a small number of users share demographics, used for training and offline evaluation. The paper compares a baseline linear model with a DNN (single hidden layer), adversarial training, and absolute correlation regularization.
- Fairness metric: conditional equality of opportunity, conditioning on feature A (e.g., averaged human rating) and weighting gaps by p_a; results are reported as FPR ratios, with error bars from training the model 10 times and averaging.
Limitations
- Stated: the paper assumes the product policy is ground truth and human raters provide unbiased estimates; rater bias is difficult to evaluate and needs further research. Sensitive attributes are unavailable at inference, and limited demographic data prevents expansive intersectional evaluation.
- Stated: evaluation uses examples currently in the system, whose distribution is affected by the system’s previous performance; only binary actions are considered, and continuous scores or composition with other signals are not addressed.
- Evident: this is a single production case study with two sensitive groups and relativistic FPR ratios; absolute correlation regularization is not provably optimal for the fairness metric, and numerical y-axis values are omitted due to sensitivity.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Own result · 4 Not checked yet · 2 Own method · 2 Own interpretation · 1 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that for both Group 1 and Group 2 the FPR Ratio > 1, with FPR Ratio Group1 = 5.3× and FPR Ratio Group2 = 2.18×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 3.1 Baseline Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we find these are sometimes unstable and difficult to reliably train well.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 1 INTRODUCTION
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe in Figure 3d that using absolute correlation regularization keeps the FPR gap approximately the same (1.05×).”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“different groups of users are associated with different types of examples, such as with different use cases or target audiences.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 4.1 Data Distribution
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that absolute correlation regularization stably and significantly improves algorithmic fairness metrics.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 1 INTRODUCTION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we see in Figure 4d, we are able to improve the FPR gap for Group 2 to 1.11×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that different ways of generating that evaluation data give significantly different results.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 4.1 Data Distribution
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we find that for both Group 1 and Group 2 the FPR Ratio > 1, with FPR Ratio Group1 = 5.3× and FPR Ratio Group2 = 2.18×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 3.1 Baseline Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we observe that the FPR does increase with the averaged human rater score”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 4.2 Distribution-Dependent Metrics
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Figure 4c we observe a very slight improvement in the FPR gap, bringing it down from 1.37× to 1.31×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
- It may state two separate claims as one. 6
- Probably not a substantial claim. 3
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we observe the FPR increases for Group 2 and Not-Group 2 usersfoot_1 ; the FPR gap decreases from 1.76× to 1.25×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Model Capacity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As can be seen in Figure 3b, the move to DNN decreases the FPR for Group 1, as well as decreasing the FPR gap from 2.62× to 1.44×.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Model Capacity.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As we see in Figure 3c, this significantly decreases the FPR gap from 1.44× to 1.04× and simultaneously decreases the FPR for Group 1, despite that not being an explicit objective.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
In the paper
“Hardt et al. [21] address this by analyzing the accuracy and asserting that the model should not mistake y = 0 examples for y = 1 examples at a higher rate for one group than another, called equality of opportunity”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 2 BACKGROUND AND RELATED WORK
What the sources say
- Not checked yetcites Hardt et al. [21]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while increasing model capacity may help in some cases, it does not necessarily improve accuracy everywhere.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Model Capacity.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Error bars are based on training the model 10 times and averaging results.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 6 IMPROVEMENTS IN PRACTICE
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both of these numbers indicate a gap in the equality of opportunity and that examples from these groups are more frequently incorrectly having adverse action taken against them.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > 3.1 Baseline Model
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The product policy is taken as ground truth.
- The authors state this as an assumption: The human raters provide an unbiased estimate of the ground truth score.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“we use only data for which y < τ , as we are concerned with the FPR.”
§ Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements > Adversarial training.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The authors state this as an assumption.
The product policy is taken as ground truth.
“We take the product policy as ground truth.”
The authors state this as an assumption.
The human raters provide an unbiased estimate of the ground truth score.
“we make the simplifying assumption that the human raters provide an unbiased estimate of the ground truth score.”
The authors state this as an assumption.
The labels are unbiased.
“This work, like most of the algorithmic fairness literature, assumes that the labels are unbiased.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, adversarial approaches have been reported to often lack stability which can make them hard to train reliably [30,99] and also speciically in some transfer learning scenarios [206] when, for example, the protected variable is known only for a small number of samples.”
From Fairness in machine learning: A survey (2024) · cites this paper as [30]
In the citing paper
“Most approaches in this area use notions of fairness within the adversary to apply feedback for model tuning as a form of in-processing where the adversary penalizes the model if a sensitive variable is predictable from the dependent variable (e.g.[28,30,58,91,289,299,305,314]).”
From Fairness in machine learning: A survey (2024) · cites this paper as [30]
In the citing paper
“Additional forms of regularization have been proposed to try and address these issues (e.g.[30]).”
From Fairness in machine learning: A survey (2024) · cites this paper as [30]
Citations
Cites
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Proceedings of the 29th International Conference on Machine Learning (ICML-12)Preprint
Langford, J. C., & Pineau, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.4676 · Cited by 1,687
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., & Erhan, D. (2016) · arXiv · doi:10.48550/arxiv.1608.06019 · Cited by 1,208
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
(2013) · Cited by 1,126
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Research works printed but not linked: 2
- 10.1145/2090236.2090255 · DOI · Paper page
- Zachary C Lipton , Alexandra Chouldechova , and Julian McAuley . 2017. Does mitigating ML's disparate impact require disparate treatment? arXiv preprint arXiv:1711.07076 ( 2017 ). Zachary C Lipton, Alexandra Chouldechova, and Julian McAuley. 2017. Does mitigating ML's disparate impact require disparate treatment? arXiv preprint arXiv:1711.07076 (2017). · arXiv
Cited by
Sorted by citations
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598
Fairness in Recommendation Ranking through Pairwise Comparisons
Beutel, A., Chen, J., Doshi, T. S., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Ed, H., et al. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330745 · Cited by 374
Self-citationBias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3631326 · Cited by 322
Fairness in Deep Learning: A Computational Perspective
Du, M., Yang, F., Zou, N., & Hu, X. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.3000681 · Cited by 218
Metadata
- Authors
- Beutel, A., Chen, J., Doshi, T., Qian, H., Woodruff, A., Luu, C., Kreitmann, P., Bischof, J., & Chi, E. H.
- Affiliations
- Google (United States) (US)
- Google, Mountain View, CA, USA
- Venue
- Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2019-01-27
- Pages
- 453-459
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W2964023221
- Cited by
- 111
- References
- 33
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 20 · 2023: 20 · 2022: 10 · 2021: 26 · 2020: 14 · 2019: 6
- FWCI
- 14.58
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- algorithmic fairness, equal opportunities, fairness metrics, distributional differences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
