The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Learning Not to Learn: Training Deep Neural Networks With Biased Data
Open accessAbstract
We propose a novel regularization algorithm to train deep neural networks, in which data at training time is severely biased. Since a neural network efficiently learns data distribution, a network is likely to learn the bias information to categorize input data. It leads to poor performance at test time, if the bias is, in fact, irrelevant to the categorization. In this paper, we formulate a regularization loss based on mutual information between feature embedding and bias. Based on the idea of minimizing this mutual information, we propose an iterative algorithm to unlearn the bias information. We employ an additional network to predict the bias distribution and train the network adversarially against the feature embedding network. At the end of learning, the bias prediction network is not able to predict the bias not because it is poorly trained, but because the feature embedding network successfully unlearns the bias information. We also demonstrate quantitative and qualitative experimental results which show that our algorithm effectively removes the bias information from feature embedding.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Age and gender classification using convolutional neural networks
Levi, G., & Hassncer, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301352 · Cited by 1,151
Simultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,149
Disentangled Representation Learning GAN for Pose-Invariant Face Recognition
Tran, L., Yin, X., & Liu, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.141 · Cited by 1,005
Age and Gender Estimation of Unfiltered Faces
Eidinger, E., Enbar, R., & Hassner, T. (2014) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2014.2359646 · Cited by 809
DEX: Deep EXpectation of Apparent Age from a Single Image
Rothe, R., Timofte, R., & Gool, L. V. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.41 · Cited by 688
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
Rotating your face using multi-task deep neural network
Yim, J., Jung, H., Yoo, B., Choi, C., Park, D., & Kim, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298667 · Cited by 275
Self-citationLearning Transferrable Representations for Unsupervised Domain Adaptation
Şener, O., Song, H. O., Saxena, A., & Savarese, S. (2016) · neural information processing systems · Cited by 244
Pairwise Confusion for Fine-Grained Visual Classification
Dubey, A., Gupta, O., Guo, P., Raskar, R., Farrell, R., & Naik, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_5 · Cited by 234
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings
Alvi, M., Zisserman, A., & Nellåker, C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11009-3_34 · Cited by 211
Beat the Machine: Challenging Humans to Find a Predictive Model's “Unknown Unknowns”
Attenberg, J., Ipeirotis, P., & Provost, F. (2015) · Journal of Data and Information Quality · doi:10.1145/2700832 · Cited by 103
Multi-task Adversarial Network for Disentangled Feature Learning
Liu, Y., Wang, Z., Jin, H., & Wassell, I. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00394 · Cited by 68
A Coverage-Based Utility Model for Identifying Unknown Unknowns
Bansal, G., & Weld, D. S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11493 · Cited by 34
Reconstruction for Feature Disentanglement in Pose-invariant Face RecognitionPreprint
Peng, X., Yu, X., Sohn, K., et al. (2017) · arXiv (Cornell University) · Cited by 18
Discovering Blind Spots of Predictive Models: Representations and Policies for Guided Exploration.Preprint
Lakkaraju, H., Kamar, E. S., Caruana, R., & Horvitz, E. (2016) · arXiv (Cornell University) · Cited by 6
Cited by
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Karkkainen, K., & Joo, J. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00159 · Cited by 527
Machine learning and deep learning—A review for ecologists
Pichler, M., & Hartig, F. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14061 · Cited by 454
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Clark, C., Yatskar, M., & Zettlemoyer, L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1418 · Cited by 381
“A popular approach is to use an adversary to remove information about a target feature, often gender or ethnicity, from a model’s internal representations (Edwards and Storkey, 2016; Wang et al., 2018; Kim et al., 2019).”
BackgroundCausal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
Feder, A., Keith, K. A., Manzoor, E., Pryzant, R., Sridhar, D., Wood-Doughty, Z., Eisenstein, J., Grimmer, J., Reichart, R., Roberts, M. E., et al. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00511 · Cited by 220
Metadata
- Authors
- Kim, B., Kim, H., Kim, K., Kim, S., & Kim, J.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 9004-9012
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963350032
- Cited by
- 345
- References
- 22
- FWCI
- 16.53
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.