A Survey on Transfer Learning
The DOI matches its Crossref record; 89 of the 90 references it cites were found in a registry or an index.
Abstract
A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These are perturbed versions of test inputs which seem almost indistinguishable from the original test input to a human, but fool the trained model [70, 441].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [70]
In the citing paper
“The rationale is to make the data distributions from different subjects more similar, and hence a classifier trained on the auxiliary data would have a better chance to perform well on the new subject.”
From Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach (2020) · cites this paper as [23]
In the citing paper
“Fortunately, transfer learning is concerned to overcome such weaknesses by applying the knowledge learned from one or multiple tasks to other related but novel ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“Transfer learning is able to get the above purpose, in which the knowledge from one or more diagnosis tasks can be reused to other related but different ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“TL has many advantages in the development of ML algorithms, such as shorter training times and the enhanced robustness of using models that have already passed initial tests.”
From A Survey of Underwater Acoustic Data Classification Methods Using Deep Learning for Shoreline Surveillance (2022) · cites this paper as [127]
14 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“Among these, a closely related learning technique to transfer learning is the multitask learning framework [21], which tries to learn multiple tasks simultaneously even when they are different.”
BackgroundTop 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · cited 1×
“ATA mining and machine learning technologies have already achieved significant success in many knowledge engineering areas including classification, regression, and clustering (e.g., [1], [2]).”
BackgroundSelf-citationCombining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753 · cited 2×
“Finally, on the WiFi localization data set, we show the comparison results reported in [67], where the baseline is a regularized least square regression model (RLSR), which is a standard regression model, and KPCA, which represents to apply RLSR on the new representations of the data learned by…”
MethodSelf-citationSemi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,505
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Richardson, M., & Domingos, P. (2006) · Machine Learning · doi:10.1007/s10994-006-5833-1 · Cited by 2,676 · cited 1×
“MLNs [56] is a powerful formalism, which combines the compact expressiveness of first-order logic with flexibility of probability, for statistical relational learning.”
BackgroundEfficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Research works printed but not linked: 3
- 1998, Learning to Learn
- 10.5555/2981562.2981582 · doi:10.5555/2981562.2981582
- Joachims, Transductive Inference for Text Classification Using Support Vector Machines, 200
Cited by
Items 601–700 of 1,257
Ngarambe, J., Yun, G. Y., & Santamouris, M. (2020) · Energy and Buildings · doi:10.1016/j.enbuild.2020.109807 · Cited by 261
Deep learning-based appearance features extraction for automated carp species identification
Banan, A., Nasiri, A., & Taheri-Garavand, A. (2020) · Aquacultural Engineering · doi:10.1016/j.aquaeng.2020.102053 · Cited by 262
A review of Machine Learning-based zero-day attack detection: Challenges and future directions
Guo, Y. (2023) · Computer Communications · doi:10.1016/j.comcom.2022.11.001 · Cited by 258
Transfer in Reinforcement Learning: A Framework and a Survey
Lazaric, A. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_5 · Cited by 256
“Unlike other learning paradigms (see Pan and Yang (2010) for a review of the possible settings in supervised learning), an RL problem is defined by different elements such as the dynamics and the reward, and the tasks in M may differ in a number of possible ways depending on the similarities and…”
BackgroundInfluentialLearning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction
Yao, H., Liu, Y., Wei, Y., Tang, X., & Li, Z. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313577 · Cited by 249
Review of machine learning methods in soft robotics
Kim, D., Kim, S.-H., Kim, T., Kang, B. B., Lee, M., Park, W., Ku, S., Kim, D., Kwon, J., Lee, H., et al. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246102 · Cited by 256
An Overview of Deep Semi-Supervised Learning
Ouali, Y., Hudelot, C., & Tami, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05278 · Cited by 230
“Transfer Learning and Domain Adaptation Transfer learning [116, 162] is used to improve a learner on one domain, called the target domain, by transferring the knowledge learned from a related domain, referred to as the source domain.”
BackgroundDeep learning for biomedical photoacoustic imaging: A review
Gröhl, J., Schellenberg, M., Dreher, K., & Maier-Hein, L. (2021) · Photoacoustics · doi:10.1016/j.pacs.2021.100241 · Cited by 248
Predictive Battery Health Management With Transfer Learning and Online Model Correction
Che, Y., Deng, Z., Lin, X., Hu, L., & Hu, X. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3055811 · Cited by 257
“and “When” to transfer the information from the source data to the target domain [34].”
BackgroundXu, C., Cao, B. T., Yuan, Y., & Meschke, G. (2023) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.115852 · Cited by 255
Goebel, R., Chander, A., Holzinger, K., Lecue, F., Akata, Z., Stumpf, S., Kieseberg, P., & Holzinger, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-99740-7_21 · Cited by 259
A survey on evolutionary machine learning
Al‐Sahaf, H., Bi, Y., Chen, Q., Lensen, A., Mei, Y., Sun, Y., Tran, B., Xue, B., & Zhang, M. (2019) · Journal of the Royal Society of New Zealand · doi:10.1080/03036758.2019.1609052 · Cited by 245
“It aims to improve the performance of learning algorithms in the target tasks/domains by using useful knowledge extracted from the source tasks/domains (Pan et al. 2010).”
BackgroundInfluentialEarly recognition of tomato gray leaf spot disease based on MobileNetv2-YOLOv3 model
Liu, J., & Wang, X. (2020) · Plant Methods · doi:10.1186/s13007-020-00624-2 · Cited by 257
Chen, J., Yi, C., Okegbile, S. D., Cai, J., & Shen, X. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3308717 · Cited by 254
“The development of general artificial intelligence (GAI) through transfer learning [266] and meta-learning [267] opens several opportunities of alleviating”
BackgroundGraph Neural Networks Exponentially Lose Expressive Power for Node Classification
Oono, K., & Suzuki, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10947 · Cited by 180
Akram, M. W., Li, G., Jin, Y., Chen, X., Zhu, C., & Ahmad, A. (2020) · Solar Energy · doi:10.1016/j.solener.2020.01.055 · Cited by 252
Comparison of Convolutional Neural Network Architectures for Classification of Tomato Plant Diseases
Maeda-Gutiérrez, V., Galván-Tejada, C. E., Zanella-Calzada, L. A., Celaya-Padilla, J. M., Galván-Tejada, J. I., Gamboa-Rosales, H., Luna-García, H., Magallanes-Quintanar, R., Guerrero Méndez, C. A., & Olvera-Olvera, C. A. (2020) · Applied Sciences · doi:10.3390/app10041245 · Cited by 255
“On the next stage, we took the advantage of Transfer Learning [28], which aims to transfer knowledge from one or more domains and apply the knowledge to another domain with a different target task.”
MethodTandel, G. S., Balestrieri, A., Jujaray, T., Khanna, N. N., Saba, L., & Suri, J. S. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103804 · Cited by 255
A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability
Zhou, Y., Wang, B., Huang, L., Cui, S., & Shao, L. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3037771 · Cited by 251
Transfer Learning With Adaptive Fine-Tuning
Vrbancic, G., & Podgorelec, V. (2020) · IEEE Access · doi:10.1109/access.2020.3034343 · Cited by 254
“The most common issue with such approaches is the assumption that the labeled and unlabeled data distributions are the same [32].”
BackgroundIdentification of rice plant diseases using lightweight attention networks
Chen, J., Zhang, D., Zeb, A., & Nanehkaran, Y. A. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114514 · Cited by 256
Deng, Y., Huang, D., Du, S., Li, G., Zhao, C., & Lv, J. (2021) · Computers in Industry · doi:10.1016/j.compind.2021.103399 · Cited by 255
A Comprehensive Survey on Source-Free Domain Adaptation
Li, J., Yu, Z., Du, Z., Zhu, L., & Shen, H. T. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3370978 · Cited by 250
“To alleviate this issue, transfer learning has been proposed as a viable solution to enable cross-domain knowledge transfer under label-deficient conditions [1], [2].”
BackgroundMengu, D., Luo, Y., Rivenson, Y., & Ozcan, A. (2020) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2921376 · Cited by 255
“dure “resembles” transfer learning, where the additional layers (and data) are used to augment the capabilities of a trained model [42].”
MethodLSDT: Latent Sparse Domain Transfer Learning for Visual Adaptation
Zhang, L., Zuo, W., & Zhang, D. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2516952 · Cited by 255
A deep neural network improves endoscopic detection of early gastric cancer without blind spots
Wu, L., Zhou, W., Wan, X., Zhang, J., Shen, L., Hu, S., Ding, Q., Mu, G., Yin, A., Huang, X., et al. (2019) · Endoscopy · doi:10.1055/a-0855-3532 · Cited by 254
Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching
Guo, Y., Gao, Y., & Shen, D. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2508280 · Cited by 251
Deep Learning for Detecting Building Defects Using Convolutional Neural Networks
Perez, H., Tah, J. H. M., & Mosavi, A. (2019) · Sensors · doi:10.3390/s19163556 · Cited by 253
Transfer fault diagnosis of bearing installed in different machines using enhanced deep auto-encoder
Zhiyi, H., Haidong, S., Lin, J., Junsheng, C., & Yu, Y. (2020) · Measurement · doi:10.1016/j.measurement.2019.107393 · Cited by 254
Cimtay, Y., & Ekmekcioglu, E. (2020) · Sensors · doi:10.3390/s20072034 · Cited by 251
Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation
Zheng, Y., Huang, D., Liu, S., & Wang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01378 · Cited by 243
A Transfer Learning approach for AI-based classification of brain tumors
Mehrotra, R., Ansari, M. A., Agrawal, R., & Anand, R. S. (2020) · Machine Learning with Applications · doi:10.1016/j.mlwa.2020.100003 · Cited by 251
Deep Learning-Enabled Semantic Communication Systems With Task-Unaware Transmitter and Dynamic Data
Zhang, H., Shao, S., Tao, M., Bi, X., & Letaief, K. B. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221991 · Cited by 252
“As a special kind of transfer learning [34], DA fulfills new tasks by transforming the samples in the source”
BackgroundRegularization for Deep Learning: A Taxonomy
Kukačka, J., Golkov, V., & Cremers, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10686 · Cited by 255
“A similar concept of sharing knowledge between tasks is also utilized in meta-learning, where multiple tasks from the same domain are learned sequentially, using previously gained knowledge as bias for new tasks (Baxter, 2000); and transfer learning, where knowledge from one domain is transferred into another domain (…”
BackgroundTransfer Learning Improves Supervised Image Segmentation Across Imaging Protocols
van Opbroek, A., Ikram, M. A., Vernooij, M. W., & de Bruijne, M. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2366792 · Cited by 253
Can Deep Learning Identify Tomato Leaf Disease?
Zhang, K., Wu, Q., Liu, A., & Meng, X. (2018) · Advances in Multimedia · doi:10.1155/2018/6710865 · Cited by 250
“Fortunately, transfer learning can improve the performance of deep neural networks by avoiding complex data mining and data-labeling efforts [28].”
BackgroundA cost effective solution for pavement crack inspection using cameras and deep neural networks
Mei, Q., & Gül, M. (2020) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2020.119397 · Cited by 253
He, Z., Shao, H., Zhong, X., & Zhao, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106396 · Cited by 250
Towards Out-Of-Distribution Generalization: A Survey
Liu, J., Shen, Z., He, Y., Zhang, X., Xu, R., Yu, H., & Cui, P. (2021) · arXiv · doi:10.48550/arxiv.2108.13624 · Cited by 253
“The detailed illustration of domain adaptation meth-ods is beyond the scope of this paper, curious readers may refer to some well established surveys [23], [24], [25], [26].”
BackgroundEvolutionary Multitasking: A Computer Science View of Cognitive Multitasking
Ong, Y.-S., & Gupta, A. (2016) · Cognitive Computation · doi:10.1007/s12559-016-9395-7 · Cited by 250
“On the other hand, if the transfer turns out to be unhelpful or negative [33], the nice property of evolution is that the negatively transferred genes get automatically eliminated from the population (over the course of a few generations) by the process of natural selection or survival of the fittest.”
BackgroundTaking Human out of Learning Applications: A Survey on Automated Machine LearningPreprint
Yao, Q., Wang, M., Escalante, H. J., Guyon, I., Hu, Y.-Q., Li, Y., Tu, W.-W., Yang, Q., & Yang, Y. (2018) · arXiv (Cornell University) · Cited by 353
Self-citationA Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06937 · Cited by 253
“Domain adaptation [61], [252] can be seen as a subset of transfer learning [253].”
BackgroundIntegrated image-based deep learning and language models for primary diabetes care
Li, J., Guan, Z., Wang, J., Cheung, C. Y., Zheng, Y., Lim, L.-L., Lim, C. C., Ruamviboonsuk, P., Raman, R., Corsino, L., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03139-8 · Cited by 250
Non-Intrusive Load Monitoring by Voltage–Current Trajectory Enabled Transfer Learning
Liu, Y., Wang, X., & You, W. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2888581 · Cited by 248
“However, transferring knowledge between two unrelated or weakly related domain could lead to bad performance, which is called ‘negative transfer’ [20].”
BackgroundDeep Transfer Learning for Few-Shot SAR Image Classification
Rostami, M., Kolouri, S., Eaton, E., & Kim, K. (2019) · Remote Sensing · doi:10.3390/rs11111374 · Cited by 247
“In the machine learning literature, challenges of learning in this scenario has been investigated within 84 transfer learning [10].”
BackgroundKumar, A., Singh, S. K., Saxena, S., Lakshmanan, K., Sangaiah, A. K., Chauhan, H., Shrivastava, S., & Singh, R. K. (2020) · Information Sciences · doi:10.1016/j.ins.2019.08.072 · Cited by 250
DCAN: Dual Channel-Wise Alignment Networks for Unsupervised Scene Adaptation
Wu, Z., Han, X., Lin, Y.-L., Uzunbas, M. G., Goldstein, T., Lim, S. N., & Davis, L. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_32 · Cited by 241
Li, Y., Li, K., Liu, X., Wang, Y., & Zhang, L. (2021) · Applied Energy · doi:10.1016/j.apenergy.2020.116410 · Cited by 247
Neupane, B., Horanont, T., & Aryal, J. (2021) · Remote Sensing · doi:10.3390/rs13040808 · Cited by 247
“The method has been quite popular in the studies that lack enough training images and labels [14,132,133].”
MethodUnderstanding Kin Relationships in a Photo
Xia, S., Shao, M., Luo, J., & Fu, Y. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2187436 · Cited by 247
An evaluation of Convolutional Neural Networks for music classification using spectrograms
Costa, Y. M. G., Oliveira, L. S., & Silla, C. N. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2016.12.024 · Cited by 246
Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
Liang, J., Hu, D., Wang, Y., He, R., & Feng, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3103390 · Cited by 241
“As a typical example of transfer learning [1], unsupervised”
BackgroundMethodSchneider, S., Taylor, G. W., Linquist, S., & Kremer, S. C. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13133 · Cited by 245
Rodrigues, T. K., Suto, K., Nishiyama, H., Liu, J., & Kato, N. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2943405 · Cited by 245
“Hou et al. [140] propose a caching strategy in MEC servers where the offered content is clustered based on its access features using k-means clustering [141] and transfer learning [211].”
BackgroundKnowledge Transfer for Rotary Machine Fault Diagnosis
Yan, R., Shen, F., Sun, C., & Chen, X. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2949057 · Cited by 245
“Although Pan and Yang [21] and Weiss et al. [4] provided a detailed survey on transfer learning in 2010 and 2016, respectively, they focused on the general introduction and engineering applications of knowledge transfer.”
BackgroundMa, K., Feng, D., Lawson, K., Tsai, W., Liang, C., Huang, X., Sharma, A., & Shen, C. (2021) · Water Resources Research · doi:10.1029/2020wr028600 · Cited by 241
“Transfer learning (TL) (Pan & Yang, 2010; Thrun & Pratt, 1998) is a method to migrate knowledge learned from one task to another.”
MethodMachine Learning Methods for Property Prediction in Chemoinformatics:Quo Vadis?
Varnek, A., & Baskin, I. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci200409x · Cited by 245
A Deep Q-Network for the Beer Game: Deep Reinforcement Learning for Inventory Optimization
Oroojlooyjadid, A., Nazari, M., Snyder, L. V., & Takáč, M. (2022) · Manufacturing & Service Operations Management · doi:10.1287/msom.2020.0939 · Cited by 240
Food and agro-product quality evaluation based on spectroscopy and deep learning: A review
Zhang, X., Yang, J., Lin, T., & Ying, Y. (2021) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2021.04.008 · Cited by 242
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Kim, B.-K., Roh, J., Dong, S.-Y., & Lee, S.-Y. (2016) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-015-0209-0 · Cited by 243
Fang, L., Wang, C., Li, S., Rabbani, H., Chen, X., & Liu, Z. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2898414 · Cited by 243
“In addition, a new CNN-based technique, known as transfer learning [42], is introduced to discriminate macular diseases with significantly fewer training OCT images, while still demonstrating promising results [43], [44].”
MethodIntegrating Machine Learning with Human Knowledge
Deng, C., Ji, X., Rainey, C., Zhang, J., & Lu, W. (2020) · iScience · doi:10.1016/j.isci.2020.101656 · Cited by 243
“Transfer learning is where knowledge is learned from a ‘‘source task’’ and applied to a ‘‘target task’’ (Pan and Yang, 2009).”
Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation
Sun, X., Wang, B., Wang, Z., Li, H., Li, H., & Fu, K. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3052869 · Cited by 239
“The main idea of transfer learning is to improve a learner in the target domain by transferring the knowledge from a related source domain [54].”
BackgroundLi, C. (2019) · Reinforcement Learning for Cyber-Physical Systems · doi:10.1201/9781351006620-6 · Cited by 188
Stable learning establishes some common ground between causal inference and machine learning
Cui, P., & Athey, S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00445-z · Cited by 241
“Transfer learning (or domain adaptation) methods 14 assume that we have previous knowledge of the target distribution that we may encounter in the test phase.”
BackgroundDeep metric learning for few-shot image classification: A Review of recent developments
Li, X., Yang, X., Ma, Z., & Xue, J.-H. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109381 · Cited by 243
“A domain consists of two components, namely a feature space X and a marginal distribution P (X) over X [18].”
BackgroundA machine learning approach to predict metabolic pathway dynamics from time-series multiomics data
Costello, Z., & Martin, H. G. (2018) · npj Systems Biology and Applications · doi:10.1038/s41540-018-0054-3 · Cited by 242
“If our final goal is to predict new biological systems, it may be more successful to look into ML techniques such as transfer learning.(80) These techniques tackle directly the challenge of predicting systems based on data originated in related systems without the need to delve into Fig.”
BackgroundBayesian multi-task learning methodology for reconstruction of structural health monitoring data
Wan, H.-P., & Ni, Y.-Q. (2019) · Structural Health Monitoring · doi:10.1177/1475921718794953 · Cited by 240
“MTL has gained a surge of research interest in the data mining and machine learning community with the purpose of classification, regression, and clustering.(27) In the MTL paradigm, a collection of related tasks are learned jointly by extracting appropriate shared information across the tasks.”
BackgroundDomain Adaptation: Challenges, Methods, Datasets, and Applications
Singhal, P., Walambe, R., Ramanna, S., & Kotecha, K. (2023) · IEEE Access · doi:10.1109/access.2023.3237025 · Cited by 234
“The seminal work on DA by Pan and Yang [6] mentions that DA is a specific case of transfer learning (TL).”
BackgroundImage-based deep learning automated sorting of date fruit
Nasiri, A., Taheri-Garavand, A., & Zhang, Y.-D. (2019) · Postharvest Biology and Technology · doi:10.1016/j.postharvbio.2019.04.003 · Cited by 240
Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation
Chen, C., Chen, Z., Jiang, B., & Jin, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013296 · Cited by 230
Measure Contribution of Participants in Federated Learning
Wang, G., Dang, C. X., & Zhou, Z. (2019) · 2019 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata47090.2019.9006179 · Cited by 221
Huo, G., Wu, Z., & Li, J. (2020) · IEEE Access · doi:10.1109/access.2020.2978880 · Cited by 241
Neural Unsupervised Domain Adaptation in NLP—A Survey
Ramponi, A., & Plank, B. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.603 · Cited by 230
Three decades of statistical pattern recognition paradigm for SHM of bridges
Figueiredo, E., & Brownjohn, J. (2022) · Structural Health Monitoring · doi:10.1177/14759217221075241 · Cited by 238
“A brief history and thorough survey on transfer learning and domain adaptation have been given in Pan and Yang [164].”
BackgroundInfluentialFrom Red Wine to Red Tomato: Composition with Context
Misra, I., Gupta, A., & Hebert, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.129 · Cited by 234
“Another area related to our work is that of transfer learning [7–9, 45, 47, 56], feature embeddings [30, 52] and lowshot recognition [14, 16, 17, 58].”
BackgroundSolving the frequency-domain acoustic VTI wave equation using physics-informed neural networks
Song, C., Alkhalifah, T., & Waheed, U. B. (2021) · Geophysical Journal International · doi:10.1093/gji/ggab010 · Cited by 239
Rating Image Aesthetics Using Deep Learning
Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477040 · Cited by 239
“Alternatively, we take ideas from inductive transfer learning [30], where we minimize the classification error with one label, while we construct feature representations with both labels.”
MethodMachine learning enables design automation of microfluidic flow-focusing droplet generation
Lashkaripour, A., Rodriguez, C., Mehdipour, N., Mardian, R., McIntyre, D., Ortiz, L., Campbell, J., & Densmore, D. (2021) · Nature Communications · doi:10.1038/s41467-020-20284-z · Cited by 236
“Second, the neural networks trained on the original data-set were utilized as base-models (pre-trained models) for automated transfer learning to allow researchers to fi ne tune the models for new fl uid combinations 42 .”
MethodZhang, Y., Hui, J., Qin, Q., Sun, Y., Zhang, T., Sun, H., & Li, M. (2021) · Remote Sensing of Environment · doi:10.1016/j.rse.2021.112724 · Cited by 236
Building energy prediction using artificial neural networks: A literature survey
Lu, C., Li, S., & Lu, Z. (2022) · Energy and Buildings · doi:10.1016/j.enbuild.2021.111718 · Cited by 236
Ding, Y., Jia, M., Zhuang, J., Cao, Y., Zhao, X., & Lee, C.-G. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108890 · Cited by 235
Billot, B., Magdamo, C., Cheng, Y., Arnold, S. E., Das, S., & Iglesias, J. E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2216399120 · Cited by 229
Demystifying Multitask Deep Neural Networks for Quantitative Structure–Activity Relationships
Xu, Y., Ma, J., Liaw, A., Sheridan, R. P., & Svetnik, V. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00087 · Cited by 238
Transfer Learning with Partial Observability Applied to Cervical Cancer Screening
Fernandes, K., Cardoso, J. S., & Fernandes, J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-58838-4_27 · Cited by 230
A Close Look at Deep Learning with Small Data
Brigato, L., & Iocchi, L. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412492 · Cited by 210
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey
Wang, Y., Yang, C., Lan, S., Zhu, L., & Zhang, Y. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3393230 · Cited by 237
“( iii) Negative transfer occurs when transferred knowledge has a detrimental impact on the target learners [183].”
Knee Point-Based Imbalanced Transfer Learning for Dynamic Multiobjective Optimization
Jiang, M., Wang, Z., Hong, H., & Yen, G. G. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2020.3004027 · Cited by 237
“…running speed is that the existing methods do not carefully select individuals during the environmental selection process, which not only results in a large amount of computing resources being wasted on low-quality individuals but also greatly increases the possibility of negative transfer [12].”
BackgroundFrom signals to knowledge: A conceptual model for multimodal learning analytics
Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12288 · Cited by 237
Reinforcement learning in sustainable energy and electric systems: a survey
Yang, T., Zhao, L., Li, W., & Zomaya, A. Y. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.03.001 · Cited by 234
Chai, R., Tsourdos, A., Savvaris, A., Chai, S., Xia, Y., & Chen, C. L. P. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3042120 · Cited by 237
Zhang, K., Cheng, H. D., & Zhang, B. (2018) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)cp.1943-5487.0000736 · Cited by 237
Chen, L., Chen, H., Wei, Z., Jin, X., Tan, X., Jin, Y., & Chen, E. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00704 · Cited by 223
A renaissance of neural networks in drug discovery
Baskin, I. I., Winkler, D., & Tetko, I. V. (2016) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2016.1201262 · Cited by 235
Investigating EEG-based functional connectivity patterns for multimodal emotion recognition
Wu, X., Zheng, W.-L., Li, Z., & Lu, B.-L. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac49a7 · Cited by 235
A Review on Deep Learning Applications in Prognostics and Health Management
Zhang, L., Lin, J., Liu, B., Zhang, Z., Yan, X., & Wei, M. (2019) · IEEE Access · doi:10.1109/access.2019.2950985 · Cited by 232
He, Z., Shao, H., Wang, P., Lin, J. (., Cheng, J., & Yang, Y. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.105313 · Cited by 235
Fan, C., Sun, Y., Xiao, F., Ma, J., Lee, D., Wang, J., & Tseng, Y. C. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.114499 · Cited by 235
White blood cells identification system based on convolutional deep neural learning networks
Shahin, A. I., Guo, Y., Amin, K. M., & Sharawi, A. A. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2017.11.015 · Cited by 232
Metadata
- Authors
- Pan, S. J., & Yang, Q.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2010-10
- Volume
- 22
- Issue
- 10
- Pages
- 1345-1359
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2165698076
- Semantic Scholar
- a25fbcbbae1e8f79c4360d26aa11a3abf1a11972
- Cited by
- 23,387
- References
- 90
- Citations by year
- 2026: 1673 · 2025: 2106 · 2024: 2284 · 2023: 2499 · 2022: 2654 · 2021: 3152 · 2020: 2705 · 2019: 2137 · 2018: 1346 · 2017: 776 · 2016: 516 · 2015: 429 · 2014: 369 · 2013: 293 · 2012: 211
- FWCI
- 160.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- transfer learning, domain adaptation, multi-task learning, sample selection bias, covariate shift, knowledge transfer, classification, regression analysis, clustering, distribution shift
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.