Analysis of Representations for Domain Adaptation
Abstract
Discriminative learning methods for classification perform well when training and test data are drawn from the same distribution. In many situations, though, we have labeled training data for a source domain, and we wish to learn a classifier which performs well on a target domain with a different distribution. Under what conditions can we adapt a classifier trained on the source domain for use in the target domain? Intuitively, a good feature representation is a crucial factor in the success of domain adaptation. We formalize this intuition theoretically with a generalization bound for domain adaption. Our theory illustrates the tradeoffs inherent in designing a representation for domain adaptation and gives a new justification for a recently proposed model. It also points toward a promising new model for domain adaptation: one which explicitly minimizes the difference between the source and target domains, while at the same time maximizing the margin of the training set. 1
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Any DL-based algorithm trained on a synthetic dataset is likely to suffer a performance drop when tested on real images, referred to as the domain gap problem.
“Any DL-based algorithm trained on a synthetic dataset is likely to suffer from a performance drop when tested on real images (whether acquired within a ground-based laboratory or in space), which is referred to as the domain gap [163] problem.”
From A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects (2023) · cites this paper as [163]
Cites
Domain adaptation with structural correspondence learning
Blitzer, J., McDonald, R., & Pereira, F. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610094 · Cited by 1,543 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · see citing works
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-009-5152-4 · Cited by 3,515 · see citing works
Self-citationGeneralizing from a Few Examples: A Survey on Few-shot Learning
Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2021) · ACM Computing Surveys · doi:10.1145/3386252 · Cited by 3,165 · see citing works
Deep domain confusion Maximizing for domain invariance
Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., & Darrell, T. (2014) · arXiv · doi:10.48550/arxiv.1412.3474 · Cited by 2,285 · see citing works
Return of Frustratingly Easy Domain Adaptation
Sun, B., Feng, J., & Saenko, K. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10306 · Cited by 2,004 · see citing works
Domain adaptation for medical image analysis: A survey
Guan, H., & Liu, M. (2022) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3117407 · Cited by 1,057 · see citing works
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046 · see citing works
Strong-Weak Distribution Alignment for Adaptive Object Detection
Saito, K., Ushiku, Y., Harada, T., & Saenko, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00712 · Cited by 819 · see citing works
Domain Generalization via Invariant Feature Representation
Muandet, K., Balduzzi, D., & Schölkopf, B. (2013) · arXiv · doi:10.48550/arxiv.1301.2115 · Cited by 771 · see citing works
Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy
Bruzzone, L., & Marconcini, M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.57 · Cited by 591 · see citing works
Domain Adaptation: Learning Bounds and Algorithms
Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009) · arXiv · doi:10.48550/arxiv.0902.3430 · Cited by 504 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
“In Appendix D, we discuss how WILDS can be used to develop and evaluate models in other problem settings that allow training algorithms to leverage additional information, such as unlabeled test data in unsupervised domain adaptation (Ben-David et al., 2006).”
Deep learning of transferable representation for scalable domain adaptation
Long, M., Wang, J., Cao, Y., Sun, J., & Yu, P. S. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2554549 · Cited by 191 · see citing works
Multi-Source Domain Adaptation with Mixture of Experts
Guo, J., Shah, D., & Barzilay, R. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1498 · Cited by 168 · see citing works
Fine-Grained Recognition in the Wild: A Multi-task Domain Adaptation Approach
Gebru, T., Hoffman, J., & Fei-Fei, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.151 · Cited by 152 · see citing works
SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap
Park, T. H., Martens, M., Lecuyer, G., Izzo, D., & D'Amico, S. (2022) · 2022 IEEE Aerospace Conference (AERO) · doi:10.1109/aero53065.2022.9843439 · Cited by 170 · see citing works
Steps Toward Robust Artificial Intelligence
Dietterich, T. G. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2756 · Cited by 139 · see citing works
On Learning Invariant Representation for Domain Adaptation
Zhao, H., Combes, R. T. D., Zhang, K., & Gordon, G. J. (2019) · arXiv · doi:10.48550/arxiv.1901.09453 · Cited by 134 · see citing works
Pauly, L., Rharbaoui, W., Shneider, C., Rathinam, A., Gaudillière, V., & Aouada, D. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.08.001 · Cited by 121 · see citing works
“Bridging the Domain Gap AnyDL-based algorithm trained on a synthetic dataset is likely to suffer from a performance drop when tested on real images (whether acquired within a ground-based laboratory or in space), which is referred to as the domain gap [9] problem.”
MethodSatellite Pose Estimation Competition 2021: Results and Analyses
Park, T. H., Märtens, M., Jawaid, M., Wang, Z., Chen, B., Chin, T.-J., Izzo, D., & D’Amico, S. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.01.002 · Cited by 91 · see citing works
One-Network Adversarial Fairness
Adel, T., Valera, I., Ghahramani, Z., & Weller, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012412 · Cited by 95 · see citing works
Support and Invertibility in Domain-Invariant Representations
Johansson, F. D., Sontag, D., & Ranganath, R. (2019) · arXiv · doi:10.48550/arxiv.1903.03448 · Cited by 70 · see citing works
A deep learning framework for hybrid heterogeneous transfer learning
Zhou, J. T., Pan, S. J., & Tsang, I. W. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.06.001 · Cited by 53 · see citing works
Liu, X.-Y., Li, G., Zhou, X.-H., Liang, X., & Hou, Z.-G. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3565754 · Cited by 36 · see citing works
A theoretical framework for deep transfer learning
Galanti, T., Wolf, L., & Hazan, T. (2016) · Information and Inference · doi:10.1093/imaiai/iaw008 · Cited by 29 · see citing works
Transferability-Guided Cross-Domain Cross-Task Transfer Learning
Tan, Y., Zhang, E., Li, Y., Huang, S.-L., & Zhang, X.-P. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3358094 · Cited by 24 · see citing works
“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”
- Authors
- Ben-David, S., Blitzer, J., Crammer, K., & Pereira, F.
- Affiliations
- University of Waterloo (CA)
- University of Pennsylvania (US)
- Venue
- Advances in Neural Information Processing Systems 19
- Publisher
- The MIT Press
- Published
- 2007-09-07
- Pages
- 137-144
- OpenAlex
- W2131953535
- Cited by
- 2,019
- References
- 1
- Citations by year
- 2026: 58 · 2025: 185 · 2024: 217 · 2023: 228 · 2022: 213 · 2021: 265 · 2020: 234 · 2019: 146 · 2018: 82 · 2017: 76 · 2016: 52 · 2015: 45 · 2014: 38 · 2013: 45 · 2012: 36
- FWCI
- 27.68
- Type
- book chapter
- Language
- en
- Open access
- closed
- Locations
- The MIT Press eBooks (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- domain adaptation, feature representation, generalization bounds, discriminative learning, classifier generalization, margin maximization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
