A Survey on Transfer Learning
The DOI matches its Crossref record; 89 of the 90 references it cites were found in a registry or an index.
Abstract
A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These are perturbed versions of test inputs which seem almost indistinguishable from the original test input to a human, but fool the trained model [70, 441].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [70]
In the citing paper
“The rationale is to make the data distributions from different subjects more similar, and hence a classifier trained on the auxiliary data would have a better chance to perform well on the new subject.”
From Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach (2020) · cites this paper as [23]
In the citing paper
“Fortunately, transfer learning is concerned to overcome such weaknesses by applying the knowledge learned from one or multiple tasks to other related but novel ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“Transfer learning is able to get the above purpose, in which the knowledge from one or more diagnosis tasks can be reused to other related but different ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“TL has many advantages in the development of ML algorithms, such as shorter training times and the enhanced robustness of using models that have already passed initial tests.”
From A Survey of Underwater Acoustic Data Classification Methods Using Deep Learning for Shoreline Surveillance (2022) · cites this paper as [127]
14 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“Among these, a closely related learning technique to transfer learning is the multitask learning framework [21], which tries to learn multiple tasks simultaneously even when they are different.”
BackgroundTop 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · cited 1×
“ATA mining and machine learning technologies have already achieved significant success in many knowledge engineering areas including classification, regression, and clustering (e.g., [1], [2]).”
BackgroundSelf-citationCombining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753 · cited 2×
“Finally, on the WiFi localization data set, we show the comparison results reported in [67], where the baseline is a regularized least square regression model (RLSR), which is a standard regression model, and KPCA, which represents to apply RLSR on the new representations of the data learned by…”
MethodSelf-citationSemi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,505
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Richardson, M., & Domingos, P. (2006) · Machine Learning · doi:10.1007/s10994-006-5833-1 · Cited by 2,676 · cited 1×
“MLNs [56] is a powerful formalism, which combines the compact expressiveness of first-order logic with flexibility of probability, for statistical relational learning.”
BackgroundEfficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Research works printed but not linked: 3
- 1998, Learning to Learn
- 10.5555/2981562.2981582 · doi:10.5555/2981562.2981582
- Joachims, Transductive Inference for Text Classification Using Support Vector Machines, 200
Cited by
Items 101–200 of 1,257
Identifying Stable Patterns over Time for Emotion Recognition from EEG
Zheng, W.-L., Zhu, J.-Y., & Lu, B.-L. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2712143 · Cited by 990
“dealing with these problems is to adopt transfer learning techniques [63], [64], [65], [66].”
BackgroundPre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994
Next-Generation Machine Learning for Biological Networks
Camacho, D. M., Collins, K. M., Powers, R. K., Costello, J. C., & Collins, J. J. (2018) · Cell · doi:10.1016/j.cell.2018.05.015 · Cited by 976
“In contrast to multi-task learning, transfer learning aims at learning a specific task from knowledge acquired while learning a different but related task (Pan and Yang, 2010).”
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
Roh, Y., Heo, G., & Whang, S. E. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2946162 · Cited by 968
Transfer learning for medical image classification: a literature review
Kim, H. E., Cosa-Linan, A., Santhanam, N., Jannesari, M., Maros, M. E., & Ganslandt, T. (2022) · BMC Medical Imaging · doi:10.1186/s12880-022-00793-7 · Cited by 965
Deep Reinforcement Learning: A Survey
Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xu, X., Dai, B., & Miao, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3207346 · Cited by 972
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987
Transfer learning using computational intelligence: A survey
Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., & Zhang, G. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.01.010 · Cited by 968
Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation
Ghifary, M., Kleijn, W. B., Zhang, M., Balduzzi, D., & Li, W. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_36 · Cited by 877
“In [23], it is considered as a special case of transfer learning.”
BackgroundMulti-Adversarial Domain Adaptation
Pei, Z., Cao, Z., Long, M., & Wang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11767 · Cited by 879
Deep Learning Algorithms for Bearing Fault Diagnostics—A Comprehensive Review
Zhang, S., Zhang, S., Wang, B., & Habetler, T. G. (2020) · IEEE Access · doi:10.1109/access.2020.2972859 · Cited by 913
“In addition, this dataset also enables the alidation of deep transfer learning and domain adaptation lgorithms [32], [33] to predict a more realistic bearing fault from accelerated life testing with classifiers trained on artificially induced scratches or drills.”
BackgroundA survey of machine learning for big data processing
Qiu, J., Wu, Q., Ding, G., Xu, Y., & Feng, S. (2016) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-016-0355-x · Cited by 894
“extract knowledge from one or more source tasks and apply the knowledge to a target task [50, 51].”
BackgroundInteractive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906
Visual Domain Adaptation: A survey of recent advances
Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2347059 · Cited by 899
“TRANSFER LEARNING Multitask learning or transfer learning is closely related to domain adaptation [7], [16].”
BackgroundTransfer Learning for Visual Categorization: A Survey
Shao, L., Zhu, F., & Li, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2330900 · Cited by 895
Wasserstein Distance Guided Representation Learning for Domain Adaptation
Shen, J., Qu, Y., Zhang, W., & Yu, Y. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11784 · Cited by 862
“Even though the source and target domains have different marginal distributions (BenDavid et al. 2007; Pan and Yang 2010), domain adaptation aims at utilizing the knowledge distilled from the source domain to help target domain learning.”
BackgroundData augmentation: A comprehensive survey of modern approaches
Mumuni, A., & Mumuni, F. (2022) · Array · doi:10.1016/j.array.2022.100258 · Cited by 891
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886
“Transfer learning was developed to address the issue whereby some representations were first pretrained on large volumes of unannotated datasets and then further adapted to guide other tasks [23].”
BackgroundMeta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Meta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
“and/or focus on algorithm selection for data mining [29], [33], [35], [36], AutoML [34], or particular applications of meta-learning such as few-shot learning [37] or NAS [38].”
InfluentialYang, B., Lei, Y., Jia, F., & Xing, S. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.12.051 · Cited by 879
Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users
Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2022.3155327 · Cited by 879
“In deep learning, it is used mostly for domain adaptation, when labeled data are abundant in a domain that is in some way similar to the domain of interest but scarce in the domain of interest [69].”
Jung, M., Schwalm, C., Migliavacca, M., Walther, S., Camps-Valls, G., Koirala, S., Anthoni, P., Besnard, S., Bodesheim, P., Carvalhais, N., et al. (2020) · Biogeosciences · doi:10.5194/bg-17-1343-2020 · Cited by 866
Transfer Learning in Deep Reinforcement Learning: A Survey
Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292075 · Cited by 861
“Alternatively, TL mechanisms from the supervised learning domain, such as meta-learning, which enables the ability of fast adaptation to new tasks [133], or importance sampling [134], which can compensate for the prior distribution changes [10], may also shed light on this question.”
BackgroundDeep Learning Based Communication Over the Air
Dorner, S., Cammerer, S., Hoydis, J., & Brink, S. T. (2018) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2784180 · Cited by 859
Computational Optimal Transport
Peyré, G., & Cuturi, M. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835519 · Cited by 847
Multi-Task Learning for Dense Prediction Tasks: A Survey
Vandenhende, S., Georgoulis, S., Van Gansbeke, W., Proesmans, M., Dai, D., & Van Gool, L. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3054719 · Cited by 840
“Transfer learning [87] makes use of the knowledge obtained when solving one task, and applies it to tackle another task.”
BackgroundYacouby, R., & Axman, D. (2020) · Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems · doi:10.18653/v1/2020.eval4nlp-1.9 · Cited by 778
“Recent developments in Transfer Learning (Pan and Yang, 2010; Conneau et al., 2020) demonstrated impressive ability to learn from small training sets (often referred to as Few-Shot Learning), showing a wide NLP community interest in improving data-efficiency during model training, but we have not found any publication…”
BackgroundA deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction
Kang, E., Min, J., & Ye, J. C. (2017) · Medical Physics · doi:10.1002/mp.12344 · Cited by 709
Li, W., Huang, R., Li, J., Liao, Y., Chen, Z., He, G., Yan, R., & Gryllias, K. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108487 · Cited by 816
Shi, Y., Larson, M., & Hanjalic, A. (2014) · ACM Computing Surveys · doi:10.1145/2556270 · Cited by 806
Strategies for Pre-training Graph Neural Networks
Hu, W., Liu, B., Gomes, J. S., Žitnik, M., Liang, P., Pande, V. S., & Leskovec, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12265 · Cited by 158
Modeling and forecasting building energy consumption: A review of data-driven techniques
Bourdeau, M., Zhai, X. Q., Nefzaoui, E., Guo, X., & Chatellier, P. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2019.101533 · Cited by 792
Transfer Learning for Low-Resource Neural Machine Translation
Zoph, B., Yuret, D., May, J., & Knight, K. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1163 · Cited by 788
Review of Image Classification Algorithms Based on Convolutional Neural Networks
Chen, L., Li, S., Bai, Q., Yang, J., Jiang, S., & Miao, Y. (2021) · Remote Sensing · doi:10.3390/rs13224712 · Cited by 789
Deep Learning in Medical Ultrasound Analysis: A Review
Liu, S., Wang, Y., Yang, X., Lei, B., Liu, L., Li, S. X., Ni, D., & Wang, T. (2019) · Engineering · doi:10.1016/j.eng.2018.11.020 · Cited by 798
Learning in Nonstationary Environments: A Survey
Ditzler, G., Roveri, M., Alippi, C., & Polikar, R. (2015) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2015.2471196 · Cited by 793
Deep Long-Tailed Learning: A Survey
Zhang, Y., Kang, B., Hooi, B., Yan, S., & Feng, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268118 · Cited by 769
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
Caballero, J., Ledig, C., Aitken, A., Acosta, A., Totz, J., Wang, Z., & Shi, W. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.304 · Cited by 763
A Survey of Optimization Methods From a Machine Learning Perspective
Sun, S., Cao, Z., Zhu, H., & Zhao, J. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2950779 · Cited by 789
“The technology of transfer learning [132] can be applied to build networks in the scenario of insufficient data.”
BackgroundSnorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3157794.3157797 · Cited by 777
“Transfer learning considers the strategy of repurposing models trained on different datasets or tasks where labeled training data is more abundant [34].”
BackgroundDeep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
“for a new but related task in the target domain without using new data [37].”
BackgroundCross-domain sentiment classification via spectral feature alignment
Pan, S. J., Ni, X., Sun, J.-T., Yang, Q., & Chen, Z. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772767 · Cited by 780
Self-citationA Survey on Federated Learning for Resource-Constrained IoT Devices
Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3095077 · Cited by 774
“SecureBoost [25] and FedBCD [26] are some of the examples of vertical FL. c) Federated transfer learning: Federated transfer learning (FTL) [27] can be considered as the combination of both horizontal and vertical partitioning of data (see Fig.”
Generalizing from a Few Examples: A Survey on Few-Shot Learning
Wang, Y., Yao, Q., Kwok, J. T.-Y., & Ni, L. M.-S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05046 · Cited by 772
“• Transfer learning [101] transfers knowledge from the source domain/task, where training data is abundant, to the target domain/task, where training data is scarce.”
BackgroundDomain Generalization via Invariant Feature Representation
Muandet, K., Balduzzi, D., & Schölkopf, B. (2013) · arXiv · doi:10.48550/arxiv.1301.2115 · Cited by 771
“Domain generalization is a form of transfer learning, which applies expertise acquired in source domains to improve learning of target domains (cf. Pan and Yang (2010a) and references therein).”
BackgroundInfluentialDeep Transfer Learning for Image‐Based Structural Damage Recognition
Gao, Y., & Mosalam, K. M. (2018) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12363 · Cited by 766
Azimi, M., Eslamlou, A., & Pekcan, G. (2020) · Sensors · doi:10.3390/s20102778 · Cited by 769
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
“…processing (NLP) (Hirschberg and Manning, 2015; Deng and Liu, 2017); robotics (Kober et al., 2013); transfer learning (Taylor and Stone, 2009; Pan and Yang, 2010; Weiss et al., 2016); semi-supervised learning (Zhu and Goldberg, 2009); Bayesian RL (Ghavamzadeh et al., 2015); spoken dialogue…”
BackgroundTransfer Joint Matching for Unsupervised Domain Adaptation
Long, M., Wang, J., Ding, G., Sun, J., & Yu, P. S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.183 · Cited by 725
Online learning: A comprehensive survey
Hoi, S. C. H., Sahoo, D., Lu, J., & Zhao, P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.112 · Cited by 746
Deep learning approaches for COVID-19 detection based on chest X-ray images
Ismael, A. M., & Şengür, A. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114054 · Cited by 746
“With a limited number of training images, DTL helps in the deep learning process for the image classification task of this proposed method (Pan & Yang, 2009).”
Ball, J. E., Anderson, D. T., & Chan, C. S. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.042609 · Cited by 746
Pham, D., Tan, X., Balderson, B., Xu, J., Grice, L. F., Yoon, S., Willis, E. F., Tran, M., Lam, P. Y., Raghubar, A., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-43120-6 · Cited by 730
Deep learning in medical imaging and radiation therapy
Sahiner, B., Pezeshk, A., Hadjiiski, L. M., Wang, X., Drukker, K., Cha, K. H., Summers, R. M., & Giger, M. L. (2019) · Medical Physics · doi:10.1002/mp.13264 · Cited by 740
Deep learning in optical metrology: a review
Zuo, C., Qian, J., Feng, S., Yin, W., Li, Y., Fan, P., Han, J., Qian, K., & Chen, Q. (2022) · Light: Science & Applications · doi:10.1038/s41377-022-00714-x · Cited by 711
A Survey of Techniques for Event Detection in Twitter
Atefeh, F., & Khreich, W. (2015) · Computational Intelligence · doi:10.1111/coin.12017 · Cited by 717
Privacy-Preserving Traffic Flow Prediction: A Federated Learning Approach
Liu, Y., Yu, J. J. Q., Kang, J., Niyato, D., & Zhang, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2991401 · Cited by 712
“[38] combined transfer learning [39] with FL to propose FedHealth to be applied to healthcare.”
BackgroundSelf-Driving Laboratories for Chemistry and Materials Science
Tom, G., Schmid, S. P., Baird, S. G., Cao, Y., Darvish, K., Hao, H., Lo, S., Pablo-García, S., Rajaonson, E. M., Skreta, M., et al. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00055 · Cited by 707
Industrial Artificial Intelligence in Industry 4.0 - Systematic Review, Challenges and Outlook
Peres, R. S., Jia, X., Lee, J., Sun, K., Colombo, A. W., & Barata, J. (2020) · IEEE Access · doi:10.1109/access.2020.3042874 · Cited by 693
“The motivation behind the concept of transfer learning lies in the fact that humans can intelligently apply previous knowledge to solve new problems either faster or with better solutions [88].”
BackgroundVariational Information Distillation for Knowledge Transfer
Ahn, S., Hu, S. X., Damianou, A., Lawrence, N. D., & Dai, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00938 · Cited by 671
“A popular approach for handling such lack of data is transfer learning [19], where the goal is to transfer knowledge from the source task to facilitate learning on the target task.”
BackgroundMachine Learning for Survival Analysis: A Survey
Wang, P., Li, Y., & Reddy, C. K. (2019) · ACM Computing Surveys · doi:10.1145/3214306 · Cited by 691
A Survey on Curriculum Learning
Wang, X., Chen, Y., & Zhu, W. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3069908 · Cited by 692
“For example, transfer learning [81] aims at alleviating the discrepancy between source tasks { T i } ni =1 and target task by transferring through model parameters of the learner.”
BackgroundVisual Domain Adaptation with Manifold Embedded Distribution Alignment
Wang, J., Feng, W., Chen, Y., Yu, H., Huang, M., & Yu, P. S. (2018) · Proceedings of the 26th ACM international conference on Multimedia · doi:10.1145/3240508.3240512 · Cited by 638
“led data to train machine learning models. „us, it is o›en necessary to leverage the abundant labeled samples in some existing domains to facilitate learning in a new target domain. Domain adaptation [27, 36] has been a promising approach to solve such cross-domain learning problems. Since the distributions of the sou…”
BackgroundIoT Security Techniques Based on Machine Learning: How Do IoT Devices Use AI to Enhance Security?
Xiao, L., Wan, X., Lu, X., Zhang, Y., & Wu, D. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2018.2825478 · Cited by 681
Transferable Representation Learning with Deep Adaptation Networks
Long, M., Cao, Y., Cao, Z., Wang, J., & Jordan, M. I. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2868685 · Cited by 676
“This work is related to transfer learning [1], which builds models that can bridge different domains or tasks, explicitly taking the cross-domain discrepancy into account.”
BackgroundOutlook on human-centric manufacturing towards Industry 5.0
Lu, Y., Zheng, H., Chand, S., Xia, W., Liu, Z., Xu, X., Wang, L., Qin, Z., & Bao, J. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.02.001 · Cited by 671
Recognition in Terra Incognita
Beery, S., Van Horn, G., & Perona, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_28 · Cited by 679
A Brief Review of Domain Adaptation
Farahani, A., Voghoei, S., Rasheed, K., & Arabnia, H. R. (2021) · Transactions on Computational Science and Computational Intelligence · doi:10.1007/978-3-030-71704-9_65 · Cited by 615
Domain adaptation for the classification of remote sensing data: An overview of recent advances
Tuia, D., Persello, C., & Bruzzone, L. (2016) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2548504 · Cited by 675
“However, we will not enter into the technical details of specific DA literature; for interested readers, we refer to the recent surveys published in [13], [14], [16], and [17].”
BackgroundMethodInfluentialA Survey on Deep Learning for Data-Driven Soft Sensors
Sun, Q., & Ge, Z. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3053128 · Cited by 665
Semantic Communication Systems for Speech Transmission
Weng, Z., & Qin, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087240 · Cited by 663
Machine Learning for the Geosciences: Challenges and Opportunities
Karpatne, A., Ebert-Uphoff, I., Ravela, S., Babaie, H. A., & Kumar, V. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2861006 · Cited by 661
Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-identification
Wang, J., Zhu, X., Gong, S., & Li, W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00242 · Cited by 637
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
Smith, J. S., Nebgen, B. T., Zubatyuk, R., Lubbers, N., Devereux, C., Barros, K., Tretiak, S., Isayev, O., & Roitberg, A. E. (2019) · Nature Communications · doi:10.1038/s41467-019-10827-4 · Cited by 664
SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining
Billot, B., Greve, D. N., Puonti, O., Thielscher, A., Van Leemput, K., Fischl, B., Dalca, A. V., & Iglesias, J. E. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102789 · Cited by 641
“While these networks obtain fast and accurate re-sults on their training domain, they do not generalise well to unseen contrasts [45] and resolutions [28], an issue known as the “domain-gap” problem [54].”
Li, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659
Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future
Nawaz, S. J., Sharma, S. K., Wyne, S., Patwary, M. N., & Asaduzzaman, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909490 · Cited by 658
“In this regard, a thorough survey of different transfer learning methods has been conducted in [103].”
BackgroundReliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization
Rahman, T., Khandakar, A., Kadir, M. A., Islam, K. R., Islam, K. F., Mazhar, R., Hamid, T., Islam, M. T., Kashem, S., Mahbub, Z. B., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3031384 · Cited by 651
Factors influencing the use of deep learning for plant disease recognition
Barbedo, J. G. A. (2018) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2018.05.013 · Cited by 649
Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation
Yan, H., Ding, Y., Li, P., Wang, Q., Xu, Y., & Zuo, W. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.107 · Cited by 613
A Decade Survey of Transfer Learning (2010–2020)
Niu, S., Liu, Y., Wang, J., & Song, H. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3054609 · Cited by 645
“The survey [2] gives readers a brief overview and detailed explanations of representative TL algorithms from 2000 to 2010.”
BackgroundMethodInfluentialRahman, T., Chowdhury, M. E. H., Khandakar, A., Islam, K. R., Islam, K. F., Mahbub, Z. B., Kadir, M. A., & Kashem, S. (2020) · Applied Sciences · doi:10.3390/app10093233 · Cited by 638
Balanced Distribution Adaptation for Transfer Learning
Wang, J., Chen, Y., Hao, S., Feng, W., & Shen, Z. (2017) · 2017 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2017.150 · Cited by 589
“In this case, transfer learning [1] has been a promising approach by transferring knowledge from a labeled source domain to the target domain.”
BackgroundYin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638
“• Instance-based FTL assumes that some labeled instances in the dataset in the source domain can be reweighted and reused for training in the target domain [11, 136].”
BackgroundMethodInfluentialSensor Technologies for Intelligent Transportation Systems
Guerrero-Ibáñez, J., Zeadally, S., & Contreras-Castillo, J. (2018) · Sensors · doi:10.3390/s18041212 · Cited by 637
“…order to: (i) perform data cleansing to remove abnormal data gathered by sources [109,110]; (ii) remove some redundant features from the original feature space but preserves the interpretability of the remaining features [111,112]; and (iii) compare and fuse the data from different sources [113].”
BackgroundAdapterFusion: Non-Destructive Task Composition for Transfer Learning
Pfeiffer, J., Kamath, A., Rücklé, A., Cho, K., & Gurevych, I. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.39 · Cited by 636
“In this section, we formalize our goal of transfer learning (Pan and Yang, 2010; Torrey and Shavlik, 2010; Ruder, 2019), highlight its key challenges, and provide a brief overview of common methods that can be used to address them.”
BackgroundTransfer Learning in Natural Language Processing
Ruder, S., Peters, M. E., Swayamdipta, S., & Wolf, T. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-5004 · Cited by 587
“The tutorial will start with a broad overview of transfer learning methods following Pan and Yang (2010).”
MethodJoint Geometrical and Statistical Alignment for Visual Domain Adaptation
Zhang, J., Li, W., & Ogunbona, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.547 · Cited by 598
“The most commonly used domain adaptation approaches include instance-based adaptation, feature representation adaptation, and classifier-based adaptation [1, 2].”
BackgroundMethodGiger (2018) · doi:10.1016/j.jacr.2017.12.028 · Cited by 634
A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme
Lao, J., Chen, Y., Li, Z.-C., Li, Q., Zhang, J., Liu, J., & Zhai, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-10649-8 · Cited by 630
Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-rays
Brunese, L., Mercaldo, F., Reginelli, A., & Santone, A. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105608 · Cited by 623
“Transfer learning typically is related to a process where a model trained on one problem is exploited with the aim to predict labels related to a second problem [23] .”
Machine Remaining Useful Life Prediction via an Attention-Based Deep Learning Approach
Chen, Z., Wu, M., Zhao, R., Guretno, F., Yan, R., & Li, X. (2021) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.2972443 · Cited by 618
AI4COVID-19: AI enabled preliminary diagnosis for COVID-19 from cough samples via an app
Imran, A., Posokhova, I., Qureshi, H. N., Masood, U., Riaz, M. S., Ali, K., John, C. N., Hussain, M. I., & Nabeel, M. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100378 · Cited by 621
“Deep transfer learning [69] is used here to transfer the knowledge (features) learned by cough detection model (trained using relatively more data) to the similar diagnosis model.”
MethodDeep-learning inversion: A next-generation seismic velocity model building method
Yang, F., & Ma, J. (2019) · GEOPHYSICS · doi:10.1190/geo2018-0249.1 · Cited by 612
A Review of Domain Adaptation without Target Labels
Kouw, W. M., & Loog, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2945942 · Cited by 613
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“This challenge arises from various practical AI tasks [343, 391] in the area of transfer learning [255, 350].”
BackgroundSliced Wasserstein Discrepancy for Unsupervised Domain Adaptation
Lee, C.-Y., Batra, T., Baig, M. H., & Ulbricht, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01053 · Cited by 577
“Given input data xs and the corresponding ground truth ys drawn from the source set {Xs, Ys}, and input data xt drawn from the target set Xt, the goal of unsupervised domain adaptation is to establish knowledge transfer from the labeled source set to the unlabeled target set as mentioned in [47].”
BackgroundRecent advances and trends in visual tracking: A review
Yang, H., Shao, L., Zheng, F., Wang, L., & Song, Z. (2011) · Neurocomputing · doi:10.1016/j.neucom.2011.07.024 · Cited by 607
“With the advances of machine learning methods such as transfer learning [130] and graphical models [131], contextual information will play an increasingly important role in future visual tracking research.”
Method
Metadata
- Authors
- Pan, S. J., & Yang, Q.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2010-10
- Volume
- 22
- Issue
- 10
- Pages
- 1345-1359
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2165698076
- Semantic Scholar
- a25fbcbbae1e8f79c4360d26aa11a3abf1a11972
- Cited by
- 23,387
- References
- 90
- Citations by year
- 2026: 1673 · 2025: 2106 · 2024: 2284 · 2023: 2499 · 2022: 2654 · 2021: 3152 · 2020: 2705 · 2019: 2137 · 2018: 1346 · 2017: 776 · 2016: 516 · 2015: 429 · 2014: 369 · 2013: 293 · 2012: 211
- FWCI
- 160.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- transfer learning, domain adaptation, multi-task learning, sample selection bias, covariate shift, knowledge transfer, classification, regression analysis, clustering, distribution shift
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.