Generating Fair Universal Representations using Adversarial Models
Open accessAbstract
We present a data-driven framework for learning fair universal representations (FUR) that guarantee statistical fairness for any learning task that may not be known a priori. Our framework leverages recent advances in adversarial learning to allow a data holder to learn representations in which a set of sensitive attributes are decoupled from the rest of the dataset. We formulate this as a constrained minimax game between an encoder and an adversary where the constraint ensures a measure of usefulness (utility) of the representation. The resulting problem is that of censoring, i.e., finding a representation that is least informative about the sensitive attributes given a utility constraint. For appropriately chosen adversarial loss functions, our censoring framework precisely clarifies the optimal adversarial strategy against strong information-theoretic adversaries; it also achieves the fairness measure of demographic parity for the resulting constrained representations. We evaluate the performance of our proposed framework on both synthetic and publicly available datasets. For these datasets, we use two tradeoff measures: censoring vs. representation fidelity and fairness vs. utility for downstream tasks, to amply demonstrate that multiple sensitive features can be effectively censored even as the resulting fair representations ensure accuracy for multiple downstream tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There has been work proposing the use of an adversary as a pre-processing transformation process on the training data with similar objectives to transformation yet often moving towards a notion of censoring the training data with similar objectives to variable blinding.
“There has also been work proposing the use of an adversary as a pre-processing transformation process on the training data (e.g.[1,99,162,206,234,299,300]) with similar objectives to transformation as discussed in subsection 4.4 yet often moving towards a notion of łcensoringž the training data with similar objectives to variable blinding as discussed in subsection 4.1.”
From Fairness in machine learning: A survey (2024) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 2× · see citing works
“Censoring is inspired by information-theoretic privacy methods to limit leakage of sensitive features [3], [16]–[19] and can be achieved in practice using generative adversarial networks (GANs) [20].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2× · see citing works
“Addressing this has broadly lead to two classes of measures: (i) group fairness measures which require similar outcomes for all groups [4]; (ii) individual fairness measures which require treating similar individuals similarly [5].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 1× · see citing works
“Addressing this has broadly lead to two classes of measures: (i) group fairness measures which require similar outcomes for all groups [4]; (ii) individual fairness measures which require treating similar individuals similarly [5].”
BackgroundBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 1× · see citing works
“It has been shown that models learned from such datasets can inherit societal bias and discrimination patterns [1], [2] and learn sensitive features even when they are not explicitly used during training [3].”
BackgroundKraskov, A., Stögbauer, H., & Grassberger, P. (2004) · Physical Review E · doi:10.1103/physreve.69.066138 · Cited by 4,157 · see citing works
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Age Progression/Regression by Conditional Adversarial Autoencoder
Zhang, Z., Song, Y., & Qi, H. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.463 · Cited by 1,480 · cited 2× · see citing works
“Approaches combining both fairness requirements have also been considered [6], [7].”
BackgroundMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · see citing works
Anguita, D., Ghio, A., Oneto, L., Parra, X., & Reyes-Ortiz, J. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35395-6_30 · Cited by 998 · see citing works
Extremal Mechanisms for Local Differential Privacy
Kairouz, P., Oh, S., & Viswanath, P. (2014) · Journal of machine learning research · doi:10.5555/2946645.2946662 · Cited by 687 · see citing works
Self-citationDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · see citing works
Censoring Representations with an AdversaryPreprint
Edwards, H., & Storkey, A. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.05897 · Cited by 563 · see citing works
Learning Adversarially Fair and Transferable RepresentationsPreprint
Madras, D., Creager, E., Pitassi, T., & Zemel, R. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06309 · Cited by 418 · see citing works
Evidence on Discrimination in Mortgage Lending
Ladd, H. (1998) · The Journal of Economic Perspectives · doi:10.1257/jep.12.2.41 · Cited by 414 · see citing works
Mordohai, P., Medioni, G., Fua, P., Ross, A., Soh, J., Deravi, F., Triglia, A., Bazin, A., Roli, F., Bigun, J., et al. (2009) · Encyclopedia of Biometrics · doi:10.1007/978-0-387-73003-5_971 · Cited by 396
Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 381 · see citing works
Gallager, R. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139626514 · Cited by 371 · see citing works
Flexibly Fair Representation Learning by DisentanglementPreprint
Creager, E., Madras, D., Jacobsen, J.-H., Weis, M. A., Swersky, K., Pitassi, T., & Zemel, R. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02589 · Cited by 186 · see citing works
iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making
Lahoti, P., Gummadi, K. P., & Weikum, G. (2019) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2019.00121 · Cited by 161 · see citing works
Context-Aware Generative Adversarial Privacy
Huang, C., Kairouz, P., Chen, X., Sankar, L., & Rajagopal, R. (2017) · Entropy · doi:10.3390/e19120656 · Cited by 156 · see citing works
Self-citationOn solving constrained optimization problems with neural networks: a penalty method approach
Lillo, W. E., Loh, M. H., Hui, S., & Żak, S. H. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.286888 · Cited by 134 · see citing works
Estimation Efficiency Under Privacy Constraints
Asoodeh, S., Díaz, M., Alajaji, F., et al. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2865558 · Cited by 101 · see citing works
Overlearning Reveals Sensitive AttributesPreprint
Song, C., & Shmatikov, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11742 · Cited by 96 · see citing works
Tunable Measures for Information Leakage and Applications to Privacy-Utility Tradeoffs
Liao, J., Kosut, O., Sankar, L., & Calmon, F. P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2935768 · Cited by 95 · see citing works
Self-citationLearning Controllable Fair RepresentationsPreprint
Song, J., Kalluri, P., Grover, A., Zhao, S., & Ermon, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04218 · Cited by 94 · see citing works
Discrimination- and privacy-aware patterns
Hajian, S., Domingo-Ferrer, J., Monreale, A., Pedreschi, D., & Giannotti, F. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0393-7 · Cited by 93 · see citing works
Privacy-Preserving Adversarial Networks
Tripathy, A., Wang, Y., & Ishwar, P. (2019) · 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2019.8919758 · Cited by 87 · see citing works
Minimax Filter: Learning to Preserve Privacy from Inference AttacksPreprint
Hamm, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.03577 · Cited by 50 · see citing works
A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalization
Sypherd, T., Díaz, M., Cava, J., et al. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3169440 · Cited by 37 · see citing works
Self-citationData Pre-Processing for Discrimination Prevention: Information-Theoretic Optimization and Analysis
Calmon, F. D. P., Wei, D., Vinzamuri, B., Ramamurthy, K. N., & Varshney, K. R. (2018) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2018.2865887 · Cited by 33 · see citing works
Discriminately decreasing discriminability with learned image filters
Whitehill, J., & Movellan, J. (2011) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247964 · Cited by 24
Flagged first, then by citations
11 of the 47 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Self-citationFairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
- Authors
- Kairouz, P., Liao, J., Huang, C., Vyas, M., Welfert, M., & Sankar, L.
- Affiliations
- Google (United States) (US)
- Arizona State University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Volume
- 17
- Pages
- 1970-1985
- ISSN
- 1556-6013 (ISSN-L)
- arXiv
- 1910.00411
- OpenAlex
- W3159744177
- Cited by
- 19
- References
- 47
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 6 · 2023: 8 · 2021: 1
- FWCI
- 4.92
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Information Forensics and Security (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1815361, 2134256, 2031799, 1350914, 2007688, 1901243)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- adversarial learning, censoring, demographic parity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
