A Survey on Transfer Learning
The DOI matches its Crossref record; 89 of the 90 references it cites were found in a registry or an index.
Abstract
A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These are perturbed versions of test inputs which seem almost indistinguishable from the original test input to a human, but fool the trained model [70, 441].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [70]
In the citing paper
“The rationale is to make the data distributions from different subjects more similar, and hence a classifier trained on the auxiliary data would have a better chance to perform well on the new subject.”
From Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach (2020) · cites this paper as [23]
In the citing paper
“Fortunately, transfer learning is concerned to overcome such weaknesses by applying the knowledge learned from one or multiple tasks to other related but novel ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“Transfer learning is able to get the above purpose, in which the knowledge from one or more diagnosis tasks can be reused to other related but different ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“TL has many advantages in the development of ML algorithms, such as shorter training times and the enhanced robustness of using models that have already passed initial tests.”
From A Survey of Underwater Acoustic Data Classification Methods Using Deep Learning for Shoreline Surveillance (2022) · cites this paper as [127]
14 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“Among these, a closely related learning technique to transfer learning is the multitask learning framework [21], which tries to learn multiple tasks simultaneously even when they are different.”
BackgroundTop 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · cited 1×
“ATA mining and machine learning technologies have already achieved significant success in many knowledge engineering areas including classification, regression, and clustering (e.g., [1], [2]).”
BackgroundSelf-citationCombining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753 · cited 2×
“Finally, on the WiFi localization data set, we show the comparison results reported in [67], where the baseline is a regularized least square regression model (RLSR), which is a standard regression model, and KPCA, which represents to apply RLSR on the new representations of the data learned by…”
MethodSelf-citationSemi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,505
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Richardson, M., & Domingos, P. (2006) · Machine Learning · doi:10.1007/s10994-006-5833-1 · Cited by 2,676 · cited 1×
“MLNs [56] is a powerful formalism, which combines the compact expressiveness of first-order logic with flexibility of probability, for statistical relational learning.”
BackgroundEfficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Research works printed but not linked: 3
- 1998, Learning to Learn
- 10.5555/2981562.2981582 · doi:10.5555/2981562.2981582
- Joachims, Transductive Inference for Text Classification Using Support Vector Machines, 200
Cited by
Items 1,001–1,100 of 1,257
A Survey of Machine Learning Techniques for Indoor Localization and Navigation Systems
Roy, P., & Chowdhury, C. (2021) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-021-01327-z · Cited by 174
Orthogonal Gradient Descent for Continual LearningPreprint
Farajtabar, M., Azizan, N., Mott, A., & Li, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07104 · Cited by 366
“, 2009), and transfer learning (Pan and Yang, 2009; Li et al., 2019a) are other related areas helpful to develop better continual learning machines that do not catastrophically forget previous experiences.”
BackgroundFeature Ensemble Plus Sample Selection: Domain Adaptation for Sentiment Classification
Xia, R., Zong, C., Hu, X., & Cambria, E. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.27 · Cited by 172
“[Pan and Yang, 2010] presented a survey on transfer learning , which categorizes transfer learning approaches in a similar manner: feature-based transfer, instance-based transfer, and model-parameter-based transfer.”
BackgroundMethodHuang, D.-A., & Wang, Y.-C. F. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.310 · Cited by 175
A Two-Stage Weighting Framework for Multi-Source Domain Adaptation
Sun, Q., Chattopadhyay, R., Panchanathan, S., & Ye, J. (2011) · neural information processing systems · Cited by 222
Li, H., Parikh, N. A., & He, L. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00491 · Cited by 174
“Motivated by knowledge of how humans learn, several learning frameworks, such as transfer learning (Pan and Yang, 2010), one-shot learning (Fei-Fei et al., 2006), and self-taught learning (Raina et al., 2007), have been proposed.”
BackgroundDeep Multitask Metric Learning for Offline Signature Verification
Soleimani, A., Araabi, B. N., & Fouladi, K. (2016) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.05.023 · Cited by 173
Using Smart City Technology to Make Healthcare Smarter
Cook, D. J., Duncan, G., Sprint, G., & Fritz, R. L. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2787688 · Cited by 173
“form using a technique called transfer learning [28], which”
MethodDetecting repeated cancer evolution from multi-region tumor sequencing data
Caravagna, G., Giarratano, Y., Ramazzotti, D., Tomlinson, I., Graham, T. A., Sanguinetti, G., & Sottoriva, A. (2018) · Nature Methods · doi:10.1038/s41592-018-0108-x · Cited by 173
Data-Driven Structural Health Monitoring Using Feature Fusion and Hybrid Deep Learning
Dang, H. V., Tran-Ngoc, H., Nguyen, T. V., Bui-Tien, T., De Roeck, G., & Nguyen, H. X. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.3034401 · Cited by 170
“For further details of the transfer learning technique, refer to [22].”
BackgroundAnomaly Detection using One-Class Neural Networks
Chalapathy, R., Menon, A. K., & Chawla, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06360 · Cited by 169
“Following the success of transfer learning [29] to obtain rich representative features, hybrid models have adopted pre-trained transfer learning models to obtain features as inputs to anomaly detection methods.”
MethodRan, M., Hu, J., Chen, Y., Chen, H., Sun, H., Zhou, J., & Zhang, Y. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.05.001 · Cited by 169
Bringing ai to edge: from deep learning’s perspective
Liu, D., Kong, H., Luo, X., Liu, W., & Subramaniam, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.04.141 · Cited by 171
“Transferability [150]: Knowledge distillation demonstrates better transferability for the small models, i.e., for a given dataset, learning the small model from knowledge distillation outperforms learning it from scratch.”
BackgroundDeep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
Omidshafiei, S., Pazis, J., Amato, C., How, J. P., & Vian, J. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.06182 · Cited by 168
“Transfer Learning (TL) aims to generalize knowledge from a set of source tasks to a target task (Pan & Yang, 2010).”
BackgroundSocial fingerprinting: detection of spambot groups through DNA-inspired behavioral modeling
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2017.2681672 · Cited by 168
“This problem is known in machine learning literature as transfer learning or inductive learning [44].”
A Deep Learning-Based Radiomics Model for Differentiating Benign and Malignant Renal Tumors
Zhou, L., Zhang, Z., Chen, Y.-C., Zhao, Z.-Y., Yin, X.-D., & Jiang, H.-B. (2019) · Translational Oncology · doi:10.1016/j.tranon.2018.10.012 · Cited by 168
“Unfortunately, the labeled medical data are inadequate and not easily available [14]; this is also the case for renal tumor data.”
BackgroundMulti-domain Neural Network Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gašić, M., Mrkšić, N., Rojas-Barahona, L. M., Su, P.-H., Vandyke, D., & Young, S. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1015 · Cited by 167
Transfer learning in heterogeneous collaborative filtering domains
Pan, W., & Yang, Q. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2013.01.003 · Cited by 167
Self-citationChen, Z., Xu, H., Jiang, P., Yu, S., Lin, G., Bychkov, I., Hmelnov, A., Ruzhnikov, G., Zhu, N., & Liu, Z. (2021) · Journal of Hydrology · doi:10.1016/j.jhydrol.2021.126573 · Cited by 165
A data-driven approach to referable diabetic retinopathy detection
Pires, R., Avila, S., Wainer, J., Valle, E., Abramoff, M. D., & Rocha, A. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.03.009 · Cited by 164
Cross-City Transfer Learning for Deep Spatio-Temporal Prediction
Wang, L., Geng, X., Ma, X., Liu, F., & Yang, Q. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/262 · Cited by 164
“Transfer Learning addresses the machine learning problem when labeled training data is scarce [11].”
BackgroundSelf-citationMachine Learning for Security and the Internet of Things: The Good, the Bad, and the Ugly
Liang, F., Hatcher, W. G., Liao, W., Gao, W., & Yu, W. (2019) · IEEE Access · doi:10.1109/access.2019.2948912 · Cited by 164
Boosting Data-Driven Evolutionary Algorithm With Localized Data Generation
Li, J.-Y., Zhan, Z.-H., Wang, C., Jin, H., & Zhang, J. (2020) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2020.2979740 · Cited by 164
Frey, N. C., Akinwande, D., Jariwala, D., & Shenoy, V. B. (2020) · ACS Nano · doi:10.1021/acsnano.0c05267 · Cited by 164
Sparse identification of nonlinear dynamics for rapid model recovery
Quade, M., Abel, M., Nathan Kutz, J., & Brunton, S. L. (2018) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5027470 · Cited by 163
Exploring Deep Learning and Transfer Learning for Colonic Polyp Classification
Ribeiro, E., Uhl, A., Wimmer, G., & Häfner, M. (2016) · Computational and Mathematical Methods in Medicine · doi:10.1155/2016/6584725 · Cited by 163
“According to Pan and Yang [43], transfer learning can be defined by the following model.”
BackgroundDeep learning in computational mechanics: a review
Herrmann, L., & Kollmannsberger, S. (2024) · Computational Mechanics · doi:10.1007/s00466-023-02434-4 · Cited by 162
“Another technique, transfer learning [159, 160], aims at accelerating the NN training.”
Jiang, H., Shi, T., Bai, Z., & Huang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2899608 · Cited by 161
“Purpose, this parameter-transfer learning may also lead to better segmentation effects [49].”
BackgroundDeep Adversarial Domain Adaptation Model for Bearing Fault Diagnosis
Liu, Z.-H., Lu, B.-L., Wei, H.-L., Chen, L., Li, X.-H., & Ratsch, M. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2019.2932000 · Cited by 161
“Fortunately, the domain adaptation (DA) technique can be utilized to solve or alleviate the data incon-stancy issue (i.e., the inconsistency between the training and test data) [17], [18].”
MethodSeismic Impedance Inversion Using Fully Convolutional Residual Network and Transfer Learning
Wu, B., Meng, D., Wang, L., Liu, N., & Wang, Y. (2020) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2963106 · Cited by 161
“Many machine learning methods follow a crucial hypothesis that the data for training and for prediction must be in the same feature space, and same distribution [25].”
BackgroundDeep learning for fabrication and maturation of 3D bioprinted tissues and organs
Ng, W. L., Chan, A., Ong, Y. S., & Chua, C. K. (2020) · Virtual and Physical Prototyping · doi:10.1080/17452759.2020.1771741 · Cited by 161
Micro-attention for micro-expression recognition
Wang, C., Peng, M., Bi, T., & Chen, T. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.005 · Cited by 161
Challenges in benchmarking stream learning algorithms with real-world data
Souza, V. M. A., dos Reis, D. M., Maletzke, A. G., & Batista, G. E. A. P. A. (2020) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-020-00698-5 · Cited by 159
Deep Learning Ensemble for Hyperspectral Image Classification
Chen, Y., Wang, Y., Gu, Y., He, X., Ghamisi, P., & Jia, X. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2915259 · Cited by 159
“Transfer learning aims at transferring knowledge between the “source” space and “task” space, which is inspired by the human learning behavior [49].”
BackgroundKehl, K. L., Elmarakeby, H., Nishino, M., Van Allen, E. M., Lepisto, E. M., Hassett, M. J., Johnson, B. E., & Schrag, D. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2019.1800 · Cited by 159
Quadratic Autoencoder (Q-AE) for Low-Dose CT Denoising
Fan, F., Shan, H., Kalra, M. K., Singh, R., Qian, G., Getzin, M., Teng, Y., Hahn, J., & Wang, G. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2963248 · Cited by 159
“Transfer learning [49] is a type of machine learning techniques with which a model developed for a task can be”
BackgroundMultiple Kernel Learning-Based Transfer Regression for Electric Load Forecasting
Wu, D., Wang, B., Precup, D., & Boulet, B. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2933413 · Cited by 159
“According to whether the source domain and target domain have the same feature space and label space, there are two types of transfer learning: homogeneous transfer learning in which the source domain and the target domain have the same feature spaces, and heterogeneous transfer learning in which source domain and tar…”
BackgroundMapping Spiking Neural Networks to Neuromorphic Hardware
Balaji, A., Catthoor, F., Das, A., Wu, Y., Huynh, K., Dell'Anna, F. G., Indiveri, G., Krichmar, J. L., Dutt, N. D., & Schaafsma, S. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2951493 · Cited by 159
“Examples include the liquid state machine [27], zero-shot learning [61], one-shot learning [62], lifelong learning [63], transfer learning [64], and deep reinforcement learning [65] among others.”
BackgroundThree critical factors affecting automated image species recognition performance for camera traps
Schneider, S., Greenberg, S., Taylor, G. W., & Kremer, S. C. (2020) · Ecology and Evolution · doi:10.1002/ece3.6147 · Cited by 159
“This allows for learned filters, such as edge or color detectors, to be used by the model for a particular niche domain, without having to be relearned on limited data (Pan & Yang, 2010).”
MethodCOVID-19 detection from chest X-Ray images using Deep Learning and Convolutional Neural Networks
Makris, A., Kontopoulos, I., & Tserpes, K. (2020) · 11th Hellenic Conference on Artificial Intelligence · doi:10.1145/3411408.3411416 · Cited by 159
“The learned features can be used to solve a different but related task involving new data, which usually are of a smaller population to train a CNN from scratch [29].”
Anand, P., Singh, Y., Selwal, A., Alazab, M., Tanwar, S., & Kumar, N. (2020) · IEEE Access · doi:10.1109/access.2020.3022842 · Cited by 159
“Future Initiative: Applying transfer learning [214] to actual defense experiences reduces accidental exploration, increases the learning rate, and reduces the chances of having poor defense policies in the initial stage of learning.”
BackgroundYang, Z., Mitra, A., Liu, W., Berlowitz, D., & Yu, H. (2023) · Nature Communications · doi:10.1038/s41467-023-43715-z · Cited by 158
Decoding Covert Speech From EEG-A Comprehensive Review
Panachakel, J. T., & Ramakrishnan, A. G. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.642251 · Cited by 156
“TL is a machine learning approach in which the performance of a classifier in the target domain is improved by incorporating the knowledge learnt from a different domain (Pan and Yang, 2009; He and Wu, 2017; García-Salinas et al., 2019).”
BackgroundAudio-visual emotion fusion (AVEF): A deep efficient weighted approach
Ma, Y., Hao, Y., Chen, M., Chen, J., Lu, P., & Košir, A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.06.003 · Cited by 155
“In practice, we choose a strategy of transfer learning for tackling the lack of labeled data [38,39].”
BackgroundA survey on deep transfer learning to edge computing for mitigating the COVID-19 pandemic
Sufian, A., Ghosh, A., Sadiq, A. S., & Smarandache, F. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2020.101830 · Cited by 156
“Transfer Learning is a technique that effectively uses knowledge of an already learned model to solve another new task (possibly related or little related) with require of minimal re-training or fine-tuning [40, 41].”
MethodDeng, Y., Lu, L., Aponte, L., Angelidi, A. M., Novak, V., Karniadakis, G. E., & Mantzoros, C. S. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00480-x · Cited by 152
Chen, Z., & Liu, B. (2018) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00737ed1v01y201610aim033 · Cited by 154
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00951-3 · Cited by 153
“BERT and EMLo are deep bidirectional language models that work on transfer learning (Pan and Yang 2010) concept are pre-trained on a corpus and are fine-tuned for a new corpus (Devlin et al.”
BackgroundMethodInfluentialTracing the evolution of AI in the past decade and forecasting the emerging trends
Shao, Z., Zhao, R., Yuan, S., Ding, M., & Wang, Y. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118221 · Cited by 153
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147
“Hence, transfer learning [80] techniques can be adopted for the entire feature and sample space in a federated environment.”
BackgroundSoleimani, E., & Nazerfard, E. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.10.056 · Cited by 146
Deep learning in robotics: survey on model structures and training strategies
Karoly, A. I., Galambos, P., Kuti, J., & Rudas, I. J. (2021) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2020.3018325 · Cited by 146
“In these cases, we can use DL methods to approximate the core knowledge [85].”
BackgroundMethodAlzubaidi, L., Al-Shamma, O., Fadhel, M. A., Farhan, L., Zhang, J., & Duan, Y. (2020) · Electronics · doi:10.3390/electronics9030445 · Cited by 143
“Therefore, the transfer learning technique is currently employed to overcome the small dataset issue [54].”
MethodA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · Informatics · doi:10.3390/informatics5020027 · Cited by 143
“The introduction of such methods will reduce the risk of overfitting, but Bayesian models can naturally be adapted to hierarchical models which can naturally lead to transfer learning frameworks [96].”
BackgroundA Comprehensive Study of Activity Recognition Using Accelerometers
Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., & Craddock, I. (2018) · doi:10.20944/preprints201803.0147.v1 · Cited by 143
SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Tang, C. I., Perez-Pozuelo, I., Spathis, D., Brage, S., Wareham, N., & Mascolo, C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448112 · Cited by 140
Transfer learning in magnetic resonance brain imaging: a systematic review
Valverde, J. M., Imani, V., Abdollahzadeh, A., De Feo, R., Prakash, M., Ciszek, R., & Tohka, J. (2021) · Journal of Imaging · doi:10.3390/jimaging7040066 · Cited by 137
“Transfer learning approaches can be categorized based on the availability of labels in source and/or target domains during the optimization [9]: unsupervised transfer learning (unlabeled data), transductive (labels available only in the source domain), and inductive approaches (labels available in the target domains a…”
InfluentialSu, S., Li, W., Mou, J., Garg, A., Gao, L., & Liu, J. (2023) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2022.3204843 · Cited by 136
“Therefore, applying the model generated by virtual data for actual capacity calculation will obtain the capacity data, which obeys the feature space and distribution different from that of actual data [9], [34].”
MethodLORM: Learning to Optimize for Resource Management in Wireless Networks With Few Training Samples
Shen, Y., Shi, Y., Zhang, J., & Letaief, K. B. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2947591 · Cited by 134
“An essential assumption of most machine learning algorithms is that the training and future testing data are in the same feature space with the same distribution, i.e., the same task [33].”
BackgroundTransfer Learning and Meta Learning-Based Fast Downlink Beamforming Adaptation
Yuan, Y., Zheng, G., Wong, K.-K., Ottersten, B., & Luo, Z.-Q. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3035843 · Cited by 133
“and corresponding applications have been introduced in [27].”
BackgroundCross Validation Framework to Choose amongst Models and Datasets for Transfer Learning
Zhong, E., Fan, W., Yang, Q., Verscheure, O., & Ren, J. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15939-8_35 · Cited by 132
“Recently, a number of algorithms have been proposed to overcome the distribution shift, such as those reviewed in but not limited to [1].”
BackgroundSelf-citationTowards AI-Powered Personalization in MOOC Learning
Yu, H., Miao, C., Leung, C., & White, T. J. (2017) · npj Science of Learning · doi:10.1038/s41539-017-0016-3 · Cited by 131
Federated Transfer Reinforcement Learning for Autonomous Driving
Liang, X., Liu, Y., Chen, T., Liu, M., & Yang, Q. (2019) · arXiv · doi:10.48550/arxiv.1910.06001 · Cited by 131
“To overcome these challenges, we propose an end-to-end training process which leverages federated learning (FL, [3]) and transfer learning [4] to enable asynchronous learning of agents from different environments simultaneously.”
Self-citationClinical text classification research trends: Systematic literature review and open issues
Mujtaba, G., Shuib, L., Idris, N., Hoo, W. L., Raj, R. G., Khowaja, K., Shaikh, K., & Nweke, H. F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.09.034 · Cited by 123
Robot Learning From Randomized Simulations: A Review
Muratore, F., Ramos, F., Turk, G., Yu, W., Gienger, M., & Peters, J. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.799893 · Cited by 128
“Rooted in classification, transfer learning is categorized in several subfields by for example differentiating 1) if labeled data is available in the source or target domain, and 2) if the tasks in both domains are the same (Pan and Yang, 2010; Zhuang et al., 2021).”
BackgroundComparison of Pre-Trained CNNs for Audio Classification Using Transfer Learning
Tsalera, E., Papadakis, A., & Samarakou, M. (2021) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan10040072 · Cited by 127
“Transfer learning techniques have been used in classification, regression, and clustering problems [43] in different domains, typically including images such as in [44] with features from ImageNet images extracted to be used in the PASCAL VOC dataset.”
MethodTowards a predictor for CO2 plume migration using deep neural networks
Wen, G., Tang, M., & Benson, S. M. (2021) · International Journal of Greenhouse Gas Control · doi:10.1016/j.ijggc.2020.103223 · Cited by 124
Learning joint space–time–frequency features for EEG decoding on small labeled data.
Zhao, D., Tang, F., Si, B., & Feng, X. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.02.009 · Cited by 123
Wiens, J., Guttag, J., & Horvitz, E. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002162 · Cited by 123
The Future of False Information Detection on Social Media: New Perspectives and Trends
Guo, B., Ding, Y., Yao, L., Liang, Y., & Yu, Z. (2021) · ACM Computing Surveys · doi:10.1145/3393880 · Cited by 122
Bearing fault diagnosis based on SVD feature extraction and transfer learning classification
Shen, F., Chen, C., Yan, R., & Gao, R. X. (2015) · 2015 Prognostics and System Health Management Conference (PHM) · doi:10.1109/phm.2015.7380088 · Cited by 119
End-to-end generation of multiple-choice questions using text-to-text transfer transformer models
Rodriguez-Torrealba, R., Garcia-Lopez, E., & Garcia-Cabot, A. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118258 · Cited by 118
Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
Bach, S. H., Rodriguez, D., Liu, Y., Luo, C., Shao, H., Xia, C., Sen, S., Ratner, A., Hancock, B., Alborzi, H., et al. (2019) · Proceedings of the 2019 International Conference on Management of Data · doi:10.1145/3299869.3314036 · Cited by 118
A Social Media Based Index of Mental Well-Being in College Campuses
Bagroy, S., Kumaraguru, P., & De Choudhury, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025909 · Cited by 115
“In order to automatically identify posts relating to mental health expressions, we adopted an inductive transfer learning approach [49].”
BackgroundMethodTriplet adversarial domain adaptation for pixel-level classification of VHR remote sensing images
Yan, L., Fan, B., Liu, H., Huo, C., Xiang, S., & Pan, C. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2958123 · Cited by 115
Online and Offline Domain Adaptation for Reducing BCI Calibration Effort
Wu, D. (2017) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2608931 · Cited by 114
“A domain [25], [32] D in TL consists of a multidimensional feature space X and a marginal probability distribution P (x), i.”
BackgroundMethodInfluentialMixing up contrastive learning: Self-supervised representation learning for time series
Wickstrøm, K., Kampffmeyer, M., Mikalsen, K. Ø., & Jenssen, R. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.02.007 · Cited by 113
“Transfer learning is the practice of transferring knowledge from a source domain to a target domain (Pan and Yang, 2010).”
BackgroundBehzadi-khormouji, H., Rostami, H., Salehi, S., Derakhshande-Rishehri, T., Masoumi, M., Salemi, S., Keshavarz, A., Gholamrezanezhad, A., Assadi, M., & Batouli, A. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105162 · Cited by 112
A Timely Intervention: Tracking the Changing Meanings of Political Concepts with Word Vectors
Rodman, E. (2020) · Political Analysis · doi:10.1017/pan.2019.23 · Cited by 112
“…vision studies that attempt to train image classifiers, transfer learning has shown that gains in performance and e iciency are possible when a machine learning model is initialized with weights from a previous model, rather than starting the learning process from scratch (Pan and Yang 2010).”
BackgroundResultGraph Distillation for Action Detection with Privileged Modalities
Luo, Z., Hsieh, J.-T., Jiang, L., Niebles, J. C., & Fei-Fei, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_11 · Cited by 111
A survey of multimodal deep generative models
Suzuki, M., & Matsuo, Y. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2035253 · Cited by 109
Robust Multimodal Emotion Recognition from Conversation with Transformer-Based Crossmodality Fusion
Xie, B., Sidulova, M., & Park, C. H. (2021) · Sensors · doi:10.3390/s21144913 · Cited by 108
“Three different deep neural networks were used for feature extraction: Generative Pre-trained Transformer (GPT), WaveRNN, and FaceNet+RNN. Transfer learning is a training scheme that provides help for training small dataset [44].”
MethodGao, Y., Zhai, P., & Mosalam, K. M. (2021) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12741 · Cited by 105
Liu, T., & Abd-Elrahman, A. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.03.006 · Cited by 103
Tong, Z., Li, W., Zhang, B., Jiang, F., & Zhou, G. (2018) · IEEE Access · doi:10.1109/access.2018.2883078 · Cited by 102
“DA, as one type of research on transfer learning, attempts to adapt a machine learning model established in a source domain for use in a different but related target domain automatically [13].”
BackgroundGong, Y., Zhang, Y., Zhu, H., Lv, J., Cheng, Q., Zhang, H., He, Y., & Wang, S. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2946059 · Cited by 101
“Previous works have demonstrated that deep learning is advantageous for transfer learning tasks because the features learned by deep neural networks capture most of the meaningful information useful for classification [21].”
BackgroundA review on the self and dual interactions between machine learning and optimisation
Song, H., Triguero, I., & Özcan, E. (2019) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00185-z · Cited by 101
A survey on machine learning from few samples
Lu, J., Gong, P., Ye, J., Zhang, J., & Zhang, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109480 · Cited by 100
On Robustness and Transferability of Convolutional Neural Networks
Djolonga, J., Yung, J., Tschannen, M., Romijnders, R., Beyer, L., Kolesnikov, A., Puigcerver, J., Minderer, M., D'Amour, A., Moldovan, D., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01619 · Cited by 98
On Robustness and Transferability of Convolutional Neural Networks
Djolonga, J., Yung, J., Tschannen, M., Romijnders, R., Beyer, L., Kolesnikov, A., Puigcerver, J., Minderer, M., D'Amour, A., Moldovan, D., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.08558 · Cited by 98
Passive acoustic monitoring of animal populations with transfer learning
Dufourq, E., Batist, C., Foquet, R., & Durbach, I. (2022) · Ecological Informatics · doi:10.1016/j.ecoinf.2022.101688 · Cited by 97
“Similar issues are commonly addressed in the broader deep learning literature by using transfer learning (Pan and Yang, 2009; Weiss et al., 2016), a modelling strategy that uses a model trained on one dataset for the purposes of prediction on another dataset.”
BackgroundGLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling
Xu, X., Liu, X., Zhang, H., Wang, W., Nepal, S., Sefidgar, Y., Seo, W., Kuehn, K. S., Huckins, J. F., Morris, M. E., et al. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569485 · Cited by 96
“If so, it provides the opportunity to leverage a wide range of transfer learning techniques in the domain adaptation area [74, 95].”
Lim, G., Bellemo, V., Xie, Y., Lee, X. Q., Yip, M. Y. T., & Ting, D. S. W. (2020) · Eye and Vision · doi:10.1186/s40662-020-00182-7 · Cited by 96
“3) [97].”
PMC: A Privacy-preserving Deep Learning Model Customization Framework for Edge Computing
Liu, B., Li, Y., Liu, Y., Guo, Y., & Chen, X. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3432208 · Cited by 95
Learning Cumulatively to Become More Knowledgeable
Fei, G., Wang, S., & Liu, B. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939835 · Cited by 95
“In summary, this paper makes the following contributions:”
BackgroundCheplygina, V., de Bruijne, M., & Pluim, J. P. W. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.03.009 · Cited by 93
Domingos, L. C. F., Santos, P. E., Skelton, P. S. M., Brinkworth, R. S. A., & Sammut, K. (2022) · Sensors · doi:10.3390/s22062181 · Cited by 122
“Transfer learning (TL) [127] is a machine-learning technique that solves this issue by leveraging the reduced amount of labelled data in a domain (target domain) by transferring and reusing the model learned on a distinct, but related, domain (source domain).”
BackgroundHaghighatlari, M., Vishwakarma, G., Altarawy, D., Subramanian, R., Kota, B. U., Sonpal, A., Setlur, S., & Hachmann, J. (2020) · WIREs Computational Molecular Science · doi:10.1002/wcms.1458 · Cited by 92
“ChemML provides TL methods 29 that allow us, for example, to generate high-quality data-derived prediction models using a combination of lower-quality (cheap) training data and a small set of high-quality (expensive) data that by itself would be insufficient for this task.”
MethodAdaptive reconstruction of digital twins for machining systems: A transfer learning approach
Liu, S., Lu, Y., Zheng, P., Shen, H., & Bao, J. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102390 · Cited by 90
Measure and Improve Robustness in NLP Models: A Survey
Wang, X., Wang, H., & Yang, D. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.339 · Cited by 90
Metadata
- Authors
- Pan, S. J., & Yang, Q.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2010-10
- Volume
- 22
- Issue
- 10
- Pages
- 1345-1359
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2165698076
- Semantic Scholar
- a25fbcbbae1e8f79c4360d26aa11a3abf1a11972
- Cited by
- 23,387
- References
- 90
- Citations by year
- 2026: 1673 · 2025: 2106 · 2024: 2284 · 2023: 2499 · 2022: 2654 · 2021: 3152 · 2020: 2705 · 2019: 2137 · 2018: 1346 · 2017: 776 · 2016: 516 · 2015: 429 · 2014: 369 · 2013: 293 · 2012: 211
- FWCI
- 160.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- transfer learning, domain adaptation, multi-task learning, sample selection bias, covariate shift, knowledge transfer, classification, regression analysis, clustering, distribution shift
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.