A Survey on Transfer Learning
The DOI matches its Crossref record; 89 of the 90 references it cites were found in a registry or an index.
Abstract
A major assumption in many machine learning and data mining algorithms is that the training and future data must be in the same feature space and have the same distribution. However, in many real-world applications, this assumption may not hold. For example, we sometimes have a classification task in one domain of interest, but we only have sufficient training data in another domain of interest, where the latter data may be in a different feature space or follow a different data distribution. In such cases, knowledge transfer, if done successfully, would greatly improve the performance of learning by avoiding much expensive data-labeling efforts. In recent years, transfer learning has emerged as a new learning framework to address this problem. This survey focuses on categorizing and reviewing the current progress on transfer learning for classification, regression, and clustering problems. In this survey, we discuss the relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift. We also explore some potential future issues in transfer learning research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These are perturbed versions of test inputs which seem almost indistinguishable from the original test input to a human, but fool the trained model [70, 441].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [70]
In the citing paper
“The rationale is to make the data distributions from different subjects more similar, and hence a classifier trained on the auxiliary data would have a better chance to perform well on the new subject.”
From Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach (2020) · cites this paper as [23]
In the citing paper
“Fortunately, transfer learning is concerned to overcome such weaknesses by applying the knowledge learned from one or multiple tasks to other related but novel ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“Transfer learning is able to get the above purpose, in which the knowledge from one or more diagnosis tasks can be reused to other related but different ones [27].”
From Applications of machine learning to machine fault diagnosis: A review and roadmap (2020) · cites this paper as [27]
In the citing paper
“TL has many advantages in the development of ML algorithms, such as shorter training times and the enhanced robustness of using models that have already passed initial tests.”
From A Survey of Underwater Acoustic Data Classification Methods Using Deep Learning for Shoreline Surveillance (2022) · cites this paper as [127]
14 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“Among these, a closely related learning technique to transfer learning is the multitask learning framework [21], which tries to learn multiple tasks simultaneously even when they are different.”
BackgroundTop 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · cited 1×
“ATA mining and machine learning technologies have already achieved significant success in many knowledge engineering areas including classification, regression, and clustering (e.g., [1], [2]).”
BackgroundSelf-citationCombining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753 · cited 2×
“Finally, on the WiFi localization data set, we show the comparison results reported in [67], where the baseline is a regularized least square regression model (RLSR), which is a standard regression model, and KPCA, which represents to apply RLSR on the new representations of the data learned by…”
MethodSelf-citationSemi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,505
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736
Transductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Richardson, M., & Domingos, P. (2006) · Machine Learning · doi:10.1007/s10994-006-5833-1 · Cited by 2,676 · cited 1×
“MLNs [56] is a powerful formalism, which combines the compact expressiveness of first-order logic with flexibility of probability, for statistical relational learning.”
BackgroundEfficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Research works printed but not linked: 3
- 1998, Learning to Learn
- 10.5555/2981562.2981582 · doi:10.5555/2981562.2981582
- Joachims, Transductive Inference for Text Classification Using Support Vector Machines, 200
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
“, 2012) specialised on Transfer Learningfrom one dataset to another (e.g., Caruana, 1997; Schmidhuber, 2004; Pan and Yang, 2010).”
BackgroundDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
“Federated transfer learning [419] is another concept that considers challenging scenarios in which data parties share only a partial overlap in the user space or the feature space, and leverage existing transfer learning techniques [314] to build models collaboratively.”
BackgroundSelf-citationA Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678
“[194] W. Pan and Q. Yang, “Transfer learning in heterogeneous collaborative filtering domains,” Artif.”
BackgroundInfluentialWeiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260
“Pan [84] refers to inductive transfer learning as the case of having available labeled target domain data, transductive transfer learning as the case of having labeled source and no labeled target domain data, and unsupervised transfer learning as the case of having no labeled source and no labeled target domain data.”
BackgroundResultInfluentialFederated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
“e two institutions have a small intersection. On the other hand, due to the different businesses, only a small portion of the feature space from both parties overlaps. In this case, transfer learning [50] techniques can be applied to provide solutions for the entire sample and feature space under a federation (Figure2…”
BackgroundSelf-citationDeep learning in agriculture: A survey
Kamilaris, A., & Prenafeta-Boldú, F. X. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.02.016 · Cited by 5,133
“The highly hierarchical structure and large learning capacity of DL models allow them to perform classification and predictions particularly well, being flexible and adaptable for a wide variety of highly complex (from a data analysis perspective) challenges (Pan and Yang, 2010).”
BackgroundInfluentialDomain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753
“Domain adaptation can be considered as a special setting of transfer learning which aims at transferring shared knowledge across different but related tasks or domains [3]–[5].”
BackgroundSelf-citationLearning Transferable Features with Deep Adaptation Networks
Long, M., Cao, Y., Wang, J., & Jordan, M. I. (2015) · arXiv · doi:10.48550/arxiv.1502.02791 · Cited by 4,505
“A related literature is transfer learning (Pan & Yang, 2010), which learns a representation or classification model to bridge different domains or tasks, explicitly taking domain discrepancy into consideration.”
BackgroundSignalP 5.0 improves signal peptide predictions using deep neural networks
Almagro Armenteros, J. J., Tsirigos, K. D., Sønderby, C. K., Petersen, T. N., Winther, O., Brunak, S., von Heijne, G., & Nielsen, H. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0036-z · Cited by 4,284
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
“We are interested in the most general inductive transfer learning setting for NLP (Pan and Yang, 2010): Given a static source task T S and any target task T T with T S 6 = T T , we would like to improve performance on T T .”
BackgroundDeep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342
A Survey on Deep Transfer Learning
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., & Liu, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01424-7_27 · Cited by 3,165
Generalizing from a Few Examples: A Survey on Few-shot Learning
Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2021) · ACM Computing Surveys · doi:10.1145/3386252 · Cited by 3,165
“• Transfer learning [101] transfers knowledge from the source domain/task, where training data is abundant, to the target domain/task, where training data is scarce.”
BackgroundLearning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150
“Transfer learning aims to transfer knowledge between related source and target domains [36].”
BackgroundA survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Applications of machine learning to machine fault diagnosis: A review and roadmap
Lei, Y., Yang, B., Jiang, X., Jia, F., Li, N., & Nandi, A. K. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106587 · Cited by 2,997
An Introduction to Deep Learning for the Physical Layer
O'Shea, T., & Hoydis, J. (2017) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2017.2758370 · Cited by 2,980
“learning [64] is one appealing candidate for adapting an endto-end communications system trained on a statistical model to”
BackgroundRegion-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912
“R-CNN training is based on inductive transfer learning, using the taxonomy of Pan and Yang [37].”
MethodDeep learning and its applications to machine health monitoring
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.05.050 · Cited by 2,894
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00815-1 · Cited by 2,678
Machinery health prognostics: A systematic review from data acquisition to RUL prediction
Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2017.11.016 · Cited by 2,634
“The key idea of transfer learning [370] is to transfer the knowledge gained from the original domain to the target domain.”
BackgroundFederated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
“As an extension, transfer learning [184] can be used to increase the efficiency of training,”
MethodSelf-citationGeodesic flow kernel for unsupervised domain adaptation
Gong, B., Shi, Y., Sha, F., & Grauman, K. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247911 · Cited by 2,093
“Domain adaptation has been extensively studied in many areas, including in statistics and machine learning [26, 18, 2, 23], speech and language processing [7, 5, 21], and more recently computer vision [3, 14, 25, 20].”
BackgroundA review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update
Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aab2f2 · Cited by 2,239
Deep visual domain adaptation: A survey
Wang, M., & Deng, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.083 · Cited by 2,209
Deep Transfer Learning with Joint Adaptation NetworksPreprint
Long, M., Han, Z., Wang, J., & Jordan, M. I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06636 · Cited by 2,426
“However, this learning paradigm suffers from the shift in data distributions across different domains, which poses a major obstacle in adapting predictive models for the target task [1].”
BackgroundInfluentialResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data
Diakogiannis, F. I., Waldner, F., Caccetta, P., & Wu, C. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.01.013 · Cited by 2,153
“…DST 5) in the ISPRS competition used networks with pre-trained weights on external large data sets (e.g. Ima-geNet, Deng et al., 2009) and fine-tuning, i.e. a methodology called transfer learning (Pan and Yang, 2010, see also Penatti et al. (2015), Xie et al. (2015) for remote sensing applications).”
MethodDeep Hashing Network for Unsupervised Domain Adaptation
Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.572 · Cited by 2,171
“A survey of popular domain adaptation techniques for computer vision is provided in [41] and a more generic survey of transfer learning approaches can be found in [39].”
BackgroundHidden fluid mechanics: Learning velocity and pressure fields from flow visualizations
Raissi, M., Yazdani, A., & Karniadakis, G. E. (2020) · Science · doi:10.1126/science.aaw4741 · Cited by 2,103
Transfer Feature Learning with Joint Distribution Adaptation
Long, M., Wang, J., Ding, G., Sun, J., & Yu, P. S. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.274 · Cited by 2,071
“According to the literature survey [16], existing transfer learning methods can be roughly organized into two categories: instance reweighting [3, 4] and feature extraction.”
BackgroundNarin, A., Kaya, C., & Pamuk, Z. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00984-y · Cited by 2,054
“y=1 ey weights of models trained on different datasets are transferred to the new model [45, 46].”
BackgroundDeep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
“Transfer learning performs data annotation by leveraging labeled data from other auxiliary domains (Pan and Yang, 2010; Cook et al., 2013; Wang et al., 2017).”
BackgroundCan AI help in screening viral and COVID-19 pneumonia?
Chowdhury, M. E. H., Rahman, T., Khandakar, A., Mazhar, R., Kadir, M. A., Mahbub, Z. B., Islam, K. R., Khan, M. S., Iqbal, A., Emadi, N. A., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3010287 · Cited by 1,913
Data-Driven Intelligent Transportation Systems: A Survey
Zhang, J., Wang, F.-Y., Wang, K., Lin, W.-H., Xu, X., & Chen, C. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2158001 · Cited by 1,817
“The other method is to utilize transfer learning [118], which aims at generalizing the regularity learned from one or more data sets into other heterogeneous data sets.”
MethodSmart radio environments empowered by reconfigurable AI meta-surfaces: an idea whose time has come
Renzo, M. D., Debbah, M., Phan-Huy, D.-T., Zappone, A., Alouini, M.-S., Yuen, C., Sciancalepore, V., Alexandropoulos, G. C., Hoydis, J., Gacanin, H., et al. (2019) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-019-1438-9 · Cited by 1,717
“Transfer learning is a method that allows us to transfer the knowledge that is used in a given context to execute a given task, into a different but related context to execute another task [162].”
BackgroundDeep learning for smart manufacturing: Methods and applications
Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.01.003 · Cited by 1,740
Moment Matching for Multi-Source Domain Adaptation
Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., & Wang, B. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00149 · Cited by 1,621
“When the target domain is quickdraw, the multi-source methods perform worse than single-source and source only baselines, which indicates negative transfer [31] occurs in this case.”
BackgroundMethodInfluentialCombining satellite imagery and machine learning to predict poverty
Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016) · Science · doi:10.1126/science.aaf7894 · Cited by 1,664
Highly Accurate Machine Fault Diagnosis Using Deep Transfer Learning
Shao, S., McAleer, S., Yan, R., & Baldi, P. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2864759 · Cited by 1,635
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
A continual learning survey: Defying forgetting in classification tasks
De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1909.08383 · Cited by 1,647
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
A Survey on Deep Learning for Named Entity Recognition
Li, J., Sun, A., Han, J., & Li, C. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981314 · Cited by 1,567
“Transfer learning aims to perform a machine learning task on a target domain by taking advantage of knowledge learned from a source domain [147].”
BackgroundPre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591
(2015) · Springer eBooks · doi:10.1007/978-1-4899-7637-6 · Cited by 1,550
Deep learning-based electroencephalography analysis: a systematic review
Roy, Y., Banville, H., Albuquerque, I., Gramfort, A., Falk, T. H., & Faubert, J. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab260c · Cited by 1,532
“Third, DL facilitates the development of tasks that are less often attempted on EEG data such as generative modelling [52] and transfer learning [129].”
BackgroundUrban Computing: Concepts, Methodologies, and Applications
Zheng, Y., Capra, L., Wolfson, O., & Yang, H. (2014) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2629592 · Cited by 1,507
“Pan et al. [2013] identified traffic anomalies according to drivers’ routing behavior on an urban road network.”
BackgroundConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“7(g), from the source domain to the target domain so as to achieve better learning performance in the target domain [85].”
BackgroundAutomated Machine Learning: Methods, Systems, Challenges
(2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5 · Cited by 1,485
Deep learning in bioinformatics
Min, S., Lee, B., & Yoon, S. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw068 · Cited by 1,465
“In addition, transfer learning, which consists of pre-training with sufficient data from similar but different domains and fine-tuning with real data, has great advantages [24, 188].”
BackgroundA survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480
Cross-Stitch Networks for Multi-task Learning
Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.433 · Cited by 1,397
“hine learning. The term multi-task learning (MTL) itself has been broadly used [2 ,14 28 42 54 55] as an umbrella term to include representation learning and selection [4,13,31,37], transfer learning [39,41,56] etc. and their widespread applications in other fields, such as genomics [38], natural language processing [7…”
MethodDeep face recognition: A survey
Wang, M., & Deng, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.10.081 · Cited by 1,466
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425
“There are many other methods in the machine learning literature that aim to reduce data labeling efforts, such as transfer learning [179], domain adaptation [180]–[184], and semi-supervised learning [185].”
MethodA Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
“Transfer learning [134] is a possible solution for such scenarios.”
BackgroundA Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security
Al-Garadi, M. A., Mohamed, A., Al-Ali, A. K., Du, X., Ali, I., & Guizani, M. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988293 · Cited by 1,392
Review of Smart Meter Data Analytics: Applications, Methodologies, and Challenges
Wang, Y., Chen, Q., Hong, T., & Kang, C. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2818167 · Cited by 1,369
“How to apply the knowledge learned for other objects to the research objects using transfer learning can help us fully utilize various data [196].”
BackgroundA Closer Look at Few-shot Classification
Chen, W.-Y., Liu, Y., Kira, Z., Wang, Y.-C. F., & Huang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04232 · Cited by 174
“The problem of learning to generalize to unseen classes during training, known as few-shot classification, has attracted considerable attention Vinyals et al. (2016); Snell et al.”
BackgroundTowards Personalized Federated Learning
Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160699 · Cited by 1,182
“Transfer learning (TL) is commonly used for model personalization in non-federated settings [41].”
BackgroundSelf-citationChatting about ChatGPT: how may AI and GPT impact academia and libraries?
Lund, B. D., & Wang, T. (2023) · Library Hi Tech News · doi:10.1108/lhtn-01-2023-0009 · Cited by 1,355
Brain tumor classification using deep CNN features via transfer learning
Deepak, S., & Ameer, P. M. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103345 · Cited by 1,321
Unsupervised Domain Adaptation with Residual Transfer Networks
Long, M., Han, Z., Wang, J., & Jordan, M. I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04433 · Cited by 1,075
“In unsupervised domain adaptation problem, we are given a source domainDs = {(xsi , ysi )} ns i=1 of ns labeled examples and a target domain Dt = {xtj} nt j=1 of nt unlabeled examples.”
MethodEnd-To-End Multi-Task Learning With Attention
Liu, S., Johns, E., & Davison, A. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00197 · Cited by 1,202
“The term Multi-Task Learning (MTL) has been broadly used in machine learning [2, 8, 6, 17], with similarities to transfer learning [22, 18] and continual learning [29].”
BackgroundEmotionMeter: A Multimodal Framework for Recognizing Human Emotions
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., & Cichocki, A. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2797176 · Cited by 1,276
“Therefore, adapting emotion recognition models should be further studied in the future [76]–[80].”
BackgroundDeep Subdomain Adaptation Network for Image Classification
Zhu, Y., Zhuang, F., Wang, J., Ke, G., Chen, J., Bian, J., Xiong, H., & He, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2988928 · Cited by 1,245
“1) Domain Adaptation: Recent years have witnessed many approaches to solve the visual domain adaptation problem, which is also commonly framed as the visual data set bias problem [2], [3].”
BackgroundDomain Generalization with Adversarial Feature Learning
Li, H., Pan, S. J., Wang, S., & Kot, A. C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00566 · Cited by 1,213
“These are referred to as domain adaptation methods [28, 13, 9].”
BackgroundSelf-citationBig data deep learning: Challenges and perspectives
Chen, X.-W., & Lin, X. (2014) · IEEE Access · doi:10.1109/access.2014.2325029 · Cited by 1,255
A review of unsupervised feature learning and deep learning for time-series modeling
Längkvist, M., Karlsson, L., & Loutfi, A. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.01.008 · Cited by 1,243
“The self-taught learning framework attempts to use unlabeled data that does not share the labels of the classification task to improve classification performance [104,62].”
BackgroundDeep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240
“Another variant of one class neural network architecture Deep Support Vector Data Description (Deep SVDD) (Ruff et al. [2018a]) trains deep neural network to extract common factors of variation by closely mapping the normal data instances to the center of sphere, is shown to produce performance improvements on MNIST (…”
BackgroundBadNets: Evaluating Backdooring Attacks on Deep Neural Networks
Gu, T., Liu, K., Dolan-Gavitt, B., & Garg, S. (2019) · IEEE Access · doi:10.1109/access.2019.2909068 · Cited by 1,238
“Transfer learning builds on the idea that a DNN trained for one machine learning task can be used for other related tasks without having to incur the computational cost of training a new model from scratch [15], [44], [45].”
BackgroundGuo, L., Lei, Y., Xing, S., Yan, T., & Li, N. (2019) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2018.2877090 · Cited by 1,234
“knowledge from the source domain to solve a new but related task in the target domain [14], [15].”
BackgroundExploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images
Rahman, T., Khandakar, A., Qiblawey, Y., Tahir, A., Kiranyaz, S., Abul Kashem, S. B., Islam, M. T., Al Maadeed, S., Zughaier, S. M., Khan, M. S., et al. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104319 · Cited by 1,147
“Thus, it opens the opportunity of utilizing smaller datasets and also reduces the time required to develop a deep learning algorithm from scratch [24,25].”
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00064 · Cited by 1,142
“Transfer Learning [39] is commonly used in computer vision to leverage features and representations learned on one task to offset data scarcity for a related task.”
BackgroundInfluentialClassification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network
Abbas, A., Abdelsamea, M. M., & Gaber, M. M. (2021) · Applied Intelligence · doi:10.1007/s10489-020-01829-7 · Cited by 1,124
An overview of multi-task learning
Zhang, Y., & Yang, Q. (2018) · National Science Review · doi:10.1093/nsr/nwx105 · Cited by 1,117
“MTL is related to other areas in machine learning, including transfer learning [2],multi-label learning [3] andmulti-output regression, but exhibits different characteristics.”
BackgroundSelf-citationAn Embarrassingly Simple Approach to Zero-Shot Learning
Romera-Paredes, B., & Torr, P. H. S. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-50077-5_2 · Cited by 1,003
“An extensive review of transfer learning methods can be found in [29].”
BackgroundA New Deep Transfer Learning Based on Sparse Auto-Encoder for Fault Diagnosis
Wen, L., Gao, L., & Li, X. (2019) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2754287 · Cited by 1,095
“The survey paper [28] provided the related applications of TL.”
BackgroundCore microbiomes for sustainable agroecosystems
Toju, H., Peay, K. G., Yamamichi, M., Narisawa, K., Hiruma, K., Naito, K., Fukuda, S., Ushio, M., Nakaoka, S., Onoda, Y., et al. (2018) · Nature Plants · doi:10.1038/s41477-018-0139-4 · Cited by 1,077
“By unifying multiple lines of those monitoring data, AI-based informatics (for example, several machine learning methods 111 and transfer learning 115 ) and novel mathematical meth-ods for tracking biological community stability 75 will allow us to explore best timing for introducing plants with…”
BackgroundMachine-learning-guided directed evolution for protein engineering
Yang, K. K., Wu, Z., & Arnold, F. H. (2019) · Nature Methods · doi:10.1038/s41592-019-0496-6 · Cited by 1,130
Deep learning classifiers for hyperspectral imaging: A review
Paoletti, M. E., Haut, J. M., Plaza, J., & Plaza, A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.09.006 · Cited by 1,073
“, 2015b) is based on the assumption that learned features in one task can be used for other tasks (Pan and Yang, 2010).”
BackgroundDomain adaptation for object recognition: An unsupervised approach
Gopalan, R., Li, R., & Chellappa, R. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126344 · Cited by 1,053
“Of the several schools of thought addressing this problem, two prominent ones are transfer learning (TL) [32], and domain adaptation (DA) [4].”
BackgroundImitation Learning: A Survey of Learning Methods
Hussein, A., Gaber, M. M., Elyan, E., & Jayne, C. (2018) · ACM Computing Surveys · doi:10.1145/3054912 · Cited by 1,060
“Even if the agent encounters negative transfer [Pan and Yang 2010], for example, from overfitting to the source task, it can recover by learning through experience and rectifying its model in the current task to converge in an appropriate time [Kuhlmann and Stone 2007].”
BackgroundDomain adaptation for medical image analysis: A survey
Guan, H., & Liu, M. (2022) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3117407 · Cited by 1,057
“In a typical transfer learning setting, there are two concepts: “domain” and “task” [16]–[18].”
Yang, C., Zhao, H., Bruzzone, L., Benediktsson, J. A., Liang, Y., Liu, B., Zeng, X., Guan, R., Li, C., & Ouyang, Z. (2020) · Nature Communications · doi:10.1038/s41467-020-20215-y · Cited by 1,052
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046
Recent Advances in Open Set Recognition: A Survey
Geng, C., Huang, S.-J., & Chen, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2981604 · Cited by 1,006
“To meet this challenge, several related research topics have been explored including lifelong learning [1], [2], transfer learning [3], [4], [5], domain adaptation [6], [7], zero-shot [8], [9], [10], one-shot (few-shot) [11], [12], [13], [14], [15], [16], [17], [18], [19], [20] recognition/learning, open set recogniti…”
BackgroundBig Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58558-7_29 · Cited by 1,039
“A network is trained once on a large, generic dataset, and its weights are then used to initialize subsequent tasks which can be solved with fewer data points, and less compute [40,44,14].”
MethodBig Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2019) · arXiv · doi:10.48550/arxiv.1912.11370 · Cited by 1,039
“A network is trained once on a large, generic dataset, and its weights are then used to initialize subsequent tasks which can be solved with fewer data points, and less compute [40,44,14].”
MethodConcepts of Artificial Intelligence for Computer-Assisted Drug Discovery
Yang, X., Wang, Y., Byrne, R., Schneider, G., & Yang, S. (2019) · Chemical Reviews · doi:10.1021/acs.chemrev.8b00728 · Cited by 1,022
ProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009
“According to the transfer-learning paradigm, a model is first pre-trained on one task, and then fine-tuned on other downstream tasks of interest [7–9].”
MethodOptimal Transport for Domain Adaptation
Courty, N., Flamary, R., Tuia, D., & Rakotomamonjy, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2615921 · Cited by 989
“Learning in this PDF discrepancy context is denoted as the domain adaptation problem [36].”
BackgroundMultifactorial Evolution: Toward Evolutionary Multitasking
Gupta, A., Ong, Y.-S., & Feng, L. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2458037 · Cited by 993
“In other words, while certain tasks are positively impacted by the implicit genetic transfer available during multitasking, certain other tasks may be negatively impacted [45], [46].”
BackgroundIdentifying Stable Patterns over Time for Emotion Recognition from EEG
Zheng, W.-L., Zhu, J.-Y., & Lu, B.-L. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2712143 · Cited by 990
“dealing with these problems is to adopt transfer learning techniques [63], [64], [65], [66].”
BackgroundPre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994
Next-Generation Machine Learning for Biological Networks
Camacho, D. M., Collins, K. M., Powers, R. K., Costello, J. C., & Collins, J. J. (2018) · Cell · doi:10.1016/j.cell.2018.05.015 · Cited by 976
“In contrast to multi-task learning, transfer learning aims at learning a specific task from knowledge acquired while learning a different but related task (Pan and Yang, 2010).”
Metadata
- Authors
- Pan, S. J., & Yang, Q.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2010-10
- Volume
- 22
- Issue
- 10
- Pages
- 1345-1359
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2165698076
- Semantic Scholar
- a25fbcbbae1e8f79c4360d26aa11a3abf1a11972
- Cited by
- 23,387
- References
- 90
- Citations by year
- 2026: 1673 · 2025: 2106 · 2024: 2284 · 2023: 2499 · 2022: 2654 · 2021: 3152 · 2020: 2705 · 2019: 2137 · 2018: 1346 · 2017: 776 · 2016: 516 · 2015: 429 · 2014: 369 · 2013: 293 · 2012: 211
- FWCI
- 160.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Knowledge and Data Engineering (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- transfer learning, domain adaptation, multi-task learning, sample selection bias, covariate shift, knowledge transfer, classification, regression analysis, clustering, distribution shift
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.