Wasserstein Distance Guided Representation Learning for Domain Adaptation
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Domain adaptation aims at generalizing a high-performance learner on a target domain via utilizing the knowledge distilled from a source domain which has a different but related data distribution. One solution to domain adaptation is to learn domain invariant feature representations while the learned representations should also be discriminative in prediction. To learn such representations, domain adaptation frameworks usually include a domain invariant representation learning approach to measure and reduce the domain discrepancy, as well as a discriminator for classification. Inspired by Wasserstein GAN, in this paper we propose a novel approach to learn domain invariant feature representations, namely Wasserstein Distance Guided Representation Learning (WDGRL). WDGRL utilizes a neural network, denoted by the domain critic, to estimate empirical Wasserstein distance between the source and target samples and optimizes the feature extractor network to minimize the estimated Wasserstein distance in an adversarial manner. The theoretical advantages of Wasserstein distance for domain adaptation lie in its gradient property and promising generalization bound. Empirical studies on common sentiment and image classification adaptation datasets demonstrate that our proposed WDGRL outperforms the state-of-the-art domain invariant representation learning approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 2×
“Even though the source and target domains have different marginal distributions (BenDavid et al. 2007; Pan and Yang 2010), domain adaptation aims at utilizing the knowledge distilled from the source domain to help target domain learning.”
BackgroundLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800 · cited 4×
“However (Gulrajani et al. 2017) pointed out that weight clipping will cause capacity underuse and gradient vanishing or exploding problems.”
BackgroundInfluentialWeiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260
Adversarial Discriminative Domain Adaptation
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.316 · Cited by 5,154 · cited 4×
“These adversarial adaptation methods have become a popular solution to reduce domain discrepancy through an adversarial objective with respect to a domain classifier (Ganin et al. 2016; Tzeng et al. 2017).”
BackgroundMethodInfluentialLearning Transferable Features with Deep Adaptation Networks
Long, M., Cao, Y., Wang, J., & Jordan, M. I. (2015) · arXiv · doi:10.48550/arxiv.1502.02791 · Cited by 4,505 · cited 10×
“Other domain adaptation frameworks (Bousmalis et al. 2016; Tzeng et al. 2014; Long et al. 2015; 2016; Zhuang et al. 2015) are not included in the comparison, because these frameworks focus on adaptation architecture design and all compared approaches can be easily integrated into these frameworks.”
BackgroundMethodInfluentialDeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 3×
“Feature Visualization We randomly choose the D→E domain adaptation task of Amazon review dataset and plot in Figure 2 the t-SNE visualization following (Donahue et al. 2014; Long et al. 2016) to visualize the learned feature representations.”
MethodDeep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757 · cited 10×
“…Feng, and Saenko 2016) was proposed to align the secondorder statistics of the source and target distributions with a linear transformation and (Sun and Saenko 2016) extended CORAL and proposed Deep CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep…”
BackgroundMethodInfluential
Cited by
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678
Deep visual domain adaptation: A survey
Wang, M., & Deng, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.083 · Cited by 2,209
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987
A Review of Domain Adaptation without Target Labels
Kouw, W. M., & Loog, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2945942 · Cited by 613
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518
Zhang, W., Deng, L., Zhang, L., & Wu, D. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.106004 · Cited by 427
Self-citationDomain-Specific Batch Normalization for Unsupervised Domain Adaptation
Chang, W.-G., You, T., Seo, S., Kwak, S., & Han, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00753 · Cited by 424
Harmonizing Transferability and Discriminability for Adapting Object Detectors
Chen, C., Zheng, Z., Ding, X., Huang, Y., & Dou, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00889 · Cited by 350
Cheng, C., Zhou, B., Ma, G., Wu, D., & Yuan, Y. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.05.040 · Cited by 271
Cross-Domain Detection via Graph-Induced Prototype Alignment
Xu, M., Wang, H., Ni, B., Tian, Q., & Zhang, W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01237 · Cited by 265
Self-citationA Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06937 · Cited by 253
Multi-Source Domain Adaptation with Mixture of Experts
Guo, J., Shah, D., & Barzilay, R. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1498 · Cited by 168
On Learning Invariant Representation for Domain Adaptation
Zhao, H., Combes, R. T. D., Zhang, K., & Gordon, G. J. (2019) · arXiv · doi:10.48550/arxiv.1901.09453 · Cited by 134
“y between two distributions, other distance measures can be used as 4 well, e.g., the maximum mean discrepancy (MMD) [Long et al., 2014, 2015, 2016], the Wasserstein distance [Courty et al., 2017b,a, Shen et al., 2018, Lee and Raginsky, 2018], etc. Under the theoretical framework of the H-divergence, Ganin et al. [201…”
MethodA Holistic Approach to Undesired Content Detection in the Real World
Markov, T., Zhang, C., Agarwal, S., Eloundou, T., Lee, T., Adler, S., Jiang, A., & Weng, L. (2022) · arXiv · doi:10.48550/arxiv.2208.03274 · Cited by 129
Domain-Specific Suppression for Adaptive Object Detection
Wang, Y., Zhang, R., Zhang, S., Li, M., Xia, Y., Zhang, X., & Liu, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00948 · Cited by 93
“The most widely used measurements are maximum mean discrepancy(MMD)[40][25][28][27] and Wasserstein distance[36].”
MethodSelf-Supervised Wasserstein Pseudo-Labeling for Semi-Supervised Image Classification
Taherkhani, F., Dabouei, A., Soleymani, S., Dawson, J., & Nasrabadi, N. M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01209 · Cited by 45
Semi-Supervised Deep Domain Adaptation for Deepfake Detection
Seraj, M. S., Singh, A., & Chakraborty, S. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00116 · Cited by 14
Guo, S., Wang, Y., Liu, Y., Zhang, X., & Tang, B. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107756 · Cited by 7
Metadata
- Authors
- Shen, J., Qu, Y., Zhang, W., & Yu, Y.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-29
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2770645414
- Cited by
- 862
- References
- 41
- FWCI
- 34.95
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.