The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Characterizing and Avoiding Negative Transfer
Open accessAbstract
When labeled data is scarce for a specific target task, transfer learning often offers an effective solution by utilizing data from a related source task. However, when transferring knowledge from a less related source, it may inversely hurt the target performance, a phenomenon known as negative transfer. Despite its pervasiveness, negative transfer is usually described in an informal manner, lacking rigorous definition, careful analysis, or systematic treatment. This paper proposes a formal definition of negative transfer and analyzes three important aspects thereof. Stemming from this analysis, a novel technique is proposed to circumvent negative transfer by filtering out unrelated source data. Based on adversarial networks, the technique is highly generic and can be applied to a wide range of transfer learning algorithms. The proposed approach is evaluated on six state-of-the-art deep transfer methods via experiments on four benchmark datasets with varying levels of difficulty. Empirically, the proposed method consistently improves the performance of all baseline methods and largely avoids negative transfer, even when the source data is degenerate.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Adversarial Discriminative Domain Adaptation
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.316 · Cited by 5,154
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753
Learning Transferable Features with Deep Adaptation Networks
Long, M., Cao, Y., Wang, J., & Jordan, M. I. (2015) · arXiv · doi:10.48550/arxiv.1502.02791 · Cited by 4,505
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,149
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Deep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757
Adapting visual category models to new domains
Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_16 · Cited by 2,847
Unsupervised Domain Adaptation by Backpropagation
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 2,521
Deep Hashing Network for Unsupervised Domain Adaptation
Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.572 · Cited by 2,171
Analysis of Representations for Domain Adaptation
Ben-David, S., Blitzer, J., Crammer, K., & Pereira, F. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0022 · Cited by 2,019
Correcting Sample Selection Bias by Unlabeled Data
Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., & Schölkopf, B. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0080 · Cited by 1,571
Simultaneous Deep Transfer Across Domains and Tasks
Tzeng, E., Hoffman, J., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.463 · Cited by 1,149
Taskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128
Multi-Adversarial Domain Adaptation
Pei, Z., Cao, Z., Long, M., & Wang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11767 · Cited by 879
Generate to Adapt: Aligning Domains Using Generative Adversarial Networks
Sankaranarayanan, S., Balaji, Y., Castillo, C. D., & Chellappa, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00887 · Cited by 619
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
Real, E., Shlens, J., Mazzocchi, S., Pan, X., & Vanhoucke, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.789 · Cited by 551
VisDA: The Visual Domain Adaptation Challenge
Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., & Saenko, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06924 · Cited by 504
Partial Transfer Learning with Selective Adversarial Networks
Cao, Z., Long, M., Wang, J., & Jordan, M. I. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00288 · Cited by 480
Partial Adversarial Domain Adaptation
Cao, Z., Ma, L., Long, M., & Wang, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_9 · Cited by 461
To transfer or not to transfer
Rosenstein, M. T. (2005) · neural information processing systems · Cited by 430
Why do deep convolutional networks generalize so poorly to small image transformations?
Azulay, A., & Weiss, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.12177 · Cited by 428 · cited 1×
“According to studies in [28, 1], such perturbation is enough to cause misclassification for neural networks and therefore is sufficient for our purpose.”
BackgroundDomain Adaptation with Conditional Transferable Components.
Gong, M., Zhang, K., Liu, T., Tao, D., Glymour, C., & Schölkopf, B. (2016) · PubMed · Cited by 227
Learning Bounds for Importance Weighting
Cortes, C., Mansour, Y., & Mohri, M. (2010) · Cited by 205
Duan, L., Xu, D., & Chang, S.-F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247819 · Cited by 193
A theory of transfer learning with applications to active learning
Yang, L., Hanneke, S., & Carbonell, J. (2013) · Machine Learning · doi:10.1007/s10994-012-5310-y · Cited by 172
Self-citationOn handling negative transfer and imbalanced distributions in multiple source transfer learning
Ge, L., Gao, J., Ngo, H. Q., Li, K., & Zhang, A. (2014) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11217 · Cited by 128
Generative Adversarial Nets from a Density Ratio Estimation PerspectivePreprint
Uehara, M., Sato, I., Suzuki, M., Nakayama, K., & Matsuo, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02920 · Cited by 106
Feature Selection for Transfer Learning
Uguroglu, S., & Carbonell, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23808-6_28 · Cited by 86
Self-citationAnalysis of Kernel Mean Matching under Covariate ShiftPreprint
Yu, Y., & ri, C. S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4650 · Cited by 72
Completely Heterogeneous Transfer Learning with Attention - What And What Not To Transfer
Moon, S., & Carbonell, J. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/349 · Cited by 43
Self-citation
Cited by
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678
A Decade Survey of Transfer Learning (2010–2020)
Niu, S., Liu, Y., Wang, J., & Song, H. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3054609 · Cited by 645
A Review of Domain Adaptation without Target Labels
Kouw, W. M., & Loog, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2945942 · Cited by 613
Transfer learning: a friendly introduction
Hosna, A., Merry, E., Gyalmo, J., Alom, Z., Aung, Z., & Azim, M. A. (2022) · Journal of Big Data · doi:10.1186/s40537-022-00652-w · Cited by 561
Deep Transfer Learning for Bearing Fault Diagnosis: A Systematic Review Since 2016
Chen, X., Yang, R., Xue, Y., Huang, M., Ferrero, R., & Wang, Z. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3244237 · Cited by 516
Self-citation
Metadata
- Authors
- Wang, Z., Dai, Z., Póczos, B., & Carbonell, J.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 11285-11294
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2980113592
- Cited by
- 463
- References
- 39
- FWCI
- 21.64
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.