The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
Open accessAbstract
In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables -such as gender- in visual recognition tasks. We show that trained models significantly amplify the association of target labels with gender beyond what one would expect from biased datasets. Surprisingly, we show that even when datasets are balanced such that each label co-occurs equally with each gender, learned models amplify the association between labels and gender, as much as if data had not been balanced! To mitigate this, we adopt an adversarial approach to remove unwanted features corresponding to protected variables from intermediate representations in a deep neural network - and provide a detailed analysis of its effectiveness. Experiments on two datasets: the COCO dataset (objects), and the imSitu dataset (actions), show reductions in gender bias amplification while maintaining most of the accuracy of the original models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
(2018) · Cited by 3,510
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Torralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
(2013) · Cited by 1,126
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995
Separating Style and Content with Bilinear Models
Tenenbaum, J. B., & Freeman, W. T. (2000) · Neural Computation · doi:10.1162/089976600300015349 · Cited by 840 · cited 1×
“Building models under fairness objectives is also related to feature disentangling methods [28, 22, 17, 18, 19].”
BackgroundMen Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 9×
“While previously it was shown that models amplify bias when they are required to predict gender alongside target variables [39], our empirical findings indicate that when models are not trained to predict gender, they also amplify gender bias.”
BackgroundMethodInfluentialSelf-citationUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509
Undoing the Damage of Dataset Bias
Khosla, A., Zhou, T., Malisiewicz, T., Efros, A. A., & Torralba, A. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_12 · Cited by 480
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
Women and news: A long and winding road
Ross, K., & Carter, C. (2011) · Media, Culture & Society · doi:10.1177/0163443711418272 · Cited by 345
Learning Gender-Neutral Word Embeddings
Zhao, J., Zhou, Y., Li, Z., Wang, W., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1521 · Cited by 345
Self-citationPredictive Inequity in Object DetectionPreprint
Wilson, B., Hoffman, J., & Morgenstern, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.11097 · Cited by 345
Controllable Invariance through Adversarial Feature LearningPreprint
Xie, Q., Dai, Z.-H., Du, Y., Hovy, E., & Neubig, G. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1705.11122 · Cited by 326
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
Beutel, A., Chen, J., Zhao, Z., & Chi, E. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00075 · Cited by 274
Tommasi, T., Patricia, N., Caputo, B., & Tuytelaars, T. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_2 · Cited by 245
Situation Recognition: Visual Semantic Role Labeling for Image Understanding
Yatskar, M., Zettlemoyer, L., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.597 · Cited by 242
Self-citationAdversarial Removal of Demographic Attributes from Text Data
Elazar, Y., & Goldberg, Y. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1002 · Cited by 237
Disentangling Factors of Variation for Facial Expression Recognition
Rifai, S., Bengio, Y., Courville, A., Vincent, P., & Mirza, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_58 · Cited by 215
Adaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition
Liu, X., Kumar, B. V. K. V., You, J., & Jia, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.79 · Cited by 207
Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels
Misra, I., Zitnick, C. L., Mitchell, M., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.320 · Cited by 199
Fairness Without Demographics in Repeated Loss Minimization
Hashimoto, T., Srivastava, M., Namkoong, H., & Liang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08010 · Cited by 199
Beyond Parity: Fairness Objectives for Collaborative Filtering
Yao, S., & Huang, B. (2017) · arXiv · doi:10.48550/arxiv.1705.08804 · Cited by 169
Exploring Disentangled Feature Representation Beyond Face Identification
Liu, Y., Wei, F., Shao, J., Sheng, L., Yan, J., & Wang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00222 · Cited by 163
Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study
Wu, Z., Wang, Z., Wang, Z., & Jin, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_37 · Cited by 161
Efficient privacy preserving video surveillance
Upmanyu, M., Namboodiri, A., Srinathan, K., & Jawahar, C. V. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459370 · Cited by 114
Liu, P., Zhou, J. T., Tsang, I. W.-H., Meng, Z., Han, S., & Tong, Y. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_11 · Cited by 55
Stock, P., & Cissé, M. (2017) · Cited by 34
What do Deep Networks Like to See?
Palacio, S., Folz, J., Hees, J., Raue, F., Borth, D., & Dengel, A. R. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00328 · Cited by 33
Improving Smiling Detection with Race and Gender Diversity
Ryu, H. J., Mitchell, M., & Adam, H. (2017) · Cited by 15
Kim, T., Kang, D., Pulli, K., & Choi, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00098 · Cited by 12
Learning to Succeed while Teaching to Fail: Privacy in Closed Machine Learning SystemsPreprint
Sokolić, J., Qiu, Q., Rodrigues, M. R. D., & Sapiro, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08197 · Cited by 4
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P., Ding, N., & Soricut, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00356 · Cited by 612
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
Wang, Z., Qinami, K., Karakozis, I. C., Genova, K., Nair, P., Hata, K., & Russakovsky, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00894 · Cited by 274
Demographic Bias in Biometrics: A Survey on an Emerging Challenge
Drozdowski, P., Rathgeb, C., Dantcheva, A., Damer, N., & Busch, C. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2992344 · Cited by 263
In-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99
Metadata
- Authors
- Wang, T., Zhao, J., Yatskar, M., Chang, K.-W., & Ordonez, V.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 5309-5318
- OpenAlex
- W2990751682
- Cited by
- 347
- References
- 40
- FWCI
- 44.47
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.