The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
Open accessAbstract
Computer vision models learn to perform a task by capturing relevant statistics from training data. It has been shown that models learn spurious age, gender, and race correlations when trained for seemingly unrelated tasks like activity recognition or image captioning. Various mitigation techniques have been presented to prevent models from utilizing or learning such biases. However, there has been little systematic comparison between these techniques. We design a simple but surprisingly effective visual recognition benchmark for studying bias mitigation. Using this benchmark, we provide a thorough analysis of a wide range of techniques. We highlight the shortcomings of popular adversarial training approaches for bias mitigation, propose a simple but similarly effective alternative to the inference-time Reducing Bias Amplification method of Zhao et al., and design a domain-independent training technique that outperforms all other methods. Finally, we validate our findings on the attribute classification task in the CelebA dataset, where attribute presence is known to be correlated with the gender of people in the image, and demonstrate that the proposed technique is effective at mitigating real-world gender bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
(2018) · Cited by 3,510
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · cited 1×
“In relation to the former, [5, 31] study the effect of class-imbalance on learning, while [52] reveal the surprising phenomenon of bias amplification.”
BackgroundSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Unsupervised Domain Adaptation by Backpropagation
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 2,521
Exploratory Undersampling for Class-Imbalance Learning
Liu, X.-Y., Wu, J., & Zhou, Z.-H. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007853 · Cited by 2,465
Torralba, A., & Efros, A. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995347 · Cited by 2,440
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,945
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Simultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,149
(2013) · Cited by 1,126
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 10×
“In the original setting of [52], training and test time biases are equal.”
BackgroundMethodInfluentialDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676
Diagnosing Error in Object Detectors
Hoiem, D., Chodpathumwan, Y., & Dai, Q. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33712-3_25 · Cited by 520
Undoing the Damage of Dataset Bias
Khosla, A., Zhou, T., Malisiewicz, T., Efros, A. A., & Torralba, A. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_12 · Cited by 480
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
Flexibly Fair Representation Learning by DisentanglementPreprint
Creager, E., Madras, D., Jacobsen, J.-H., Weis, M. A., Swersky, K., Pitassi, T., & Zemel, R. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02589 · Cited by 372
Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure
Saerens, M., Latinne, P., & Decaestecker, C. (2002) · Neural Computation · doi:10.1162/089976602753284446 · Cited by 361
Avoiding Discrimination through Causal Reasoning
Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., & Schölkopf, B. (2017) · arXiv · doi:10.48550/arxiv.1706.02744 · Cited by 358
Racial faces in the wild: Reducing racial bias by information maximization adaptation network
Wang, M., Deng, W., Hu, J., Tao, X., & Huang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00078 · Cited by 353
Wang, T., Zhao, J., Yatskar, M., Chang, K.-W., & Ordonez, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00541 · Cited by 347
Learning Not to Learn: Training Deep Neural Networks With Biased Data
Kim, B., Kim, H., Kim, K., Kim, S., & Kim, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00922 · Cited by 345
On Formalizing Fairness in Prediction with Machine LearningPreprint
Gajane, P., & Pechenizkiy, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03184 · Cited by 311
What makes ImageNet good for transfer learning?
Huh, M., Agrawal, P., & Efros, A. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.08614 · Cited by 266
Weiss, G. M., McCarthy, K., & Zabar, B. (2007) · DMIN · Cited by 234
Obstructing Classification via Projection
Haghighatkhah, P., Meulemans, W., Speckmann, B., Urhausen, J., & Verbeek, K. (2021) · Leibniz international proceedings in informatics · doi:10.4230/lipics.mfcs.2021.56 · Cited by 229
REPAIR: Removing Representation Bias by Dataset Resampling
Li, Y., & Vasconcelos, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00980 · Cited by 223
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings
Alvi, M., Zisserman, A., & Nellåker, C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11009-3_34 · Cited by 211
Discriminative Learning Under Covariate Shift
BickelSteffen, BrücknerMichael, & SchefferTobias (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755858 · Cited by 183
Discovering Fair Representations in the Data Domain
Quadrianto, N., Sharmanska, V., & Thomas, O. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00842 · Cited by 172
Bias Correction of Learned Generative Models using Likelihood-Free Importance WeightingPreprint
Grover, A., Song, J., Agarwal, A., Tran, K., Kapoor, A., Horvitz, E., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09531 · Cited by 133
What Actions are Needed for Understanding Human Actions in Videos?
Sigurdsson, G. A., Russakovsky, O., & Gupta, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.235 · Cited by 131
Self-citationInclusiveFaceNet: Improving Face Attribute Detection with Race and Gender DiversityPreprint
Ryu, H. J., Adam, H., & Mitchell, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00193 · Cited by 126
Fair Generative Modeling via Weak SupervisionPreprint
Choi, K., Grover, A., Singh, T., Shu, R., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12008 · Cited by 124
Detecting Avocados to Zucchinis: What Have We Done, and Where Are We Going?
Russakovsky, O., Deng, J., Huang, Z., Berg, A. C., & Fei-Fei, L. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.258 · Cited by 100
Self-citationCINIC-10 Is Not ImageNet or CIFAR-10Preprint
Darlow, L. N., Crowley, E. J., Antoniou, A., & Storkey, A. (2018) · arXiv (Cornell University) · doi:10.7488/ds/2448 · Cited by 97
Classifier adaptation at prediction time
Royer, A., & Lampert, C. H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298746 · Cited by 71
Feature-Wise Bias AmplificationPreprint
Leino, K., Black, E., Fredrikson, M., Sen, S., & Datta, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08999 · Cited by 50
Adapting Models to Signal Degradation using Distillation
Su, J.-C., & Maji, S. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.21 · Cited by 33
Improving Smiling Detection with Race and Gender Diversity
Ryu, H. J., Mitchell, M., & Adam, H. (2017) · Cited by 15
Cited by
Generative AI in medical practice: in-depth exploration of privacy and security challenges
Chen, Y., & Esmaeilzadeh, P. (2024) · Journal of Medical Internet Research · doi:10.2196/53008 · Cited by 353
“However, bias mitigation and high validation thresholds are critical before real-world adoption [28].”
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169
Salehi, M., Mirzaei, H., Hendrycks, D., Li, Y., Rohban, M. H., & Sabokrou, M. (2021) · arXiv · doi:10.48550/arxiv.2110.14051 · Cited by 118
Mitigating face recognition bias via group adaptive classifier
Gong, S., Liu, X., & Jain, A. K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00342 · Cited by 92
Metadata
- Authors
- Wang, Z., Qinami, K., Karakozis, I. C., Genova, K., Nair, P., Hata, K., & Russakovsky, O.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 8916-8925
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034700241
- Cited by
- 274
- References
- 49
- FWCI
- 41.20
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.