Transferability-Guided Cross-Domain Cross-Task Transfer Learning

“In addition, we further examine the effectiveness of our OTCE-based finetune algorithm with different model architectures including famous LeNet [59] for character recognition (MNIST → Omniglot) and the ResNet-18 [36] for natural image classification (Caltech101 → MiniImageNet).”

“In addition, we further examine the effectiveness of our OTCE-based finetune algorithm with different model architectures including famous LeNet [59] for character recognition (MNIST → Omniglot) and the ResNet-18 [36] for natural image classification (Caltech101 → MiniImageNet).”

“More specifically, the model zoo contains 15 models trained on ImageNet [39] by various algorithms, e.g., supervised learning (Sup-100%), semi-supervised learning (Semi-rotation-10% and Semi-exemplar-10% [40]), self-supervised learning (Rotation [41] and Jigsaw [42]), generative method (Cond-biggan…”

“Illustration of three different transfer learning settings, i.e., transductive domain adaptation [6], cross-task transfer [7], and cross-domain cross-task transfer we investigating. tasks are unrelated or if the transferred representation does not carry sufficient information about the target task,…”

“On the other hand, the vanilla finetune method even outperforms existing representative few-shot learning approaches including ProtoNet [55], MatchingNet [54], MAML [53], and Rela-Fig.”

“As empirical transferability studies [16], [17], [18], [19], [24] incur heavy computational burdens in retraining the transfer learning model on the target training data, a new 2162-237X © 2024 IEEE.”

“…of transferability metrics, we adopt the commonly used Spearman’s rank correlation coefficient (Spearman’s ρ coefficient) and the Kendall rank correlation coefficient (Kendall’s τ coefficient) [35] to assess the correlation between the transfer accuracy and predicted transferability scores.”

“1) Task Generation: Specifically, we generate few-shot target tasks using the character recognition dataset Omniglot [57] and the natural image classification dataset MiniIma-geNet [54], which are commonly used benchmarks in few-shot learning.”

“…model zoo contains 15 models trained on ImageNet [39] by various algorithms, e.g., supervised learning (Sup-100%), semi-supervised learning (Semi-rotation-10% and Semi-exemplar-10% [40]), self-supervised learning (Rotation [41] and Jigsaw [42]), generative method (Cond-biggan [43]), and VAEs [44].”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“Mean-while, VTAB provides the transfer accuracy of these models on seven target datasets including Caltech101 [45], CIFAR-100 [46], DTD [47], Flowers102 [48], Pets [49], SVHN [50], and Camelyon [51].”

“Mean-while, VTAB provides the transfer accuracy of these models on seven target datasets including Caltech101 [45], CIFAR-100 [46], DTD [47], Flowers102 [48], Pets [49], SVHN [50], and Camelyon [51].”

“1) Datasets: Our experiments are conducted on the data from the largest-to-date cross-domain dataset DomainNet [33] and popular Office31 [34] dataset.”

“…model zoo contains 15 models trained on ImageNet [39] by various algorithms, e.g., supervised learning (Sup-100%), semi-supervised learning (Semi-rotation-10% and Semi-exemplar-10% [40]), self-supervised learning (Rotation [41] and Jigsaw [42]), generative method (Cond-biggan [43]), and VAEs [44].”

“…model zoo contains 15 models trained on ImageNet [39] by various algorithms, e.g., supervised learning (Sup-100%), semi-supervised learning (Semi-rotation-10% and Semi-exemplar-10% [40]), self-supervised learning (Rotation [41] and Jigsaw [42]), generative method (Cond-biggan [43]), and VAEs [44].”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“1) Datasets: Our experiments are conducted on the data from the largest-to-date cross-domain dataset DomainNet [33] and popular Office31 [34] dataset.”

“T RANSFER learning is an effective learning paradigm to enhance the performance on target tasks via leveraging prior knowledge from the related source tasks (or source models), especially when there are only a few labeled data for supervision [1], [2], [3], [4].”

“…of the OTCE-based URL algorithm on the popular metadataset [60] benchmark, which contains eight training domains (datasets), such as ImageNet [39], Omniglot [57], and Aircraft [61] and five testing domains including TrafficSign [62], MS-COCO [63], MNIST [58], CIFAR-10 [46], and CIFAR-100 [46].”

“…of the OTCE-based URL algorithm on the popular metadataset [60] benchmark, which contains eight training domains (datasets), such as ImageNet [39], Omniglot [57], and Aircraft [61] and five testing domains including TrafficSign [62], MS-COCO [63], MNIST [58], CIFAR-10 [46], and CIFAR-100 [46].”

“T RANSFER learning is an effective learning paradigm to enhance the performance on target tasks via leveraging prior knowledge from the related source tasks (or source models), especially when there are only a few labeled data for supervision [1], [2], [3], [4].”

“…model zoo contains 15 models trained on ImageNet [39] by various algorithms, e.g., supervised learning (Sup-100%), semi-supervised learning (Semi-rotation-10% and Semi-exemplar-10% [40]), self-supervised learning (Rotation [41] and Jigsaw [42]), generative method (Cond-biggan [43]), and VAEs [44].”

“Therefore, we propose to replace the knowledge distillation objective centered kernel alignment (CKA) [32] similarity used in URL with our F-OTCE metric, as illustrated in Fig.”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“Inthisarticle, we mainly investigate the transferability estimation problem with two representative transfer paradigms for neural networks [5], i.e., Retrain head and Finetune .”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“G. Evaluation on OTCE-Based URL Algorithm 1) Dataset: We conduct the evaluations of the OTCE-based URL algorithm on the popular metadataset [60] benchmark, which contains eight training domains (datasets), such as ImageNet [39], Omniglot [57], and Aircraft [61] and five testing domains including…”

“The computation of solving the OT problem with entropic regularizer [28] (4) is differentiable [30] since the iterations form a sequence of linear operations, so it can be implemented on the PyTorch framework as a specialized layer 1 of the neural network.”

“T RANSFER learning is an effective learning paradigm to enhance the performance on target tasks via leveraging prior knowledge from the related source tasks (or source models), especially when there are only a few labeled data for supervision [1], [2], [3], [4].”

“In this exper-iment, we evaluate the effectiveness of transferability metrics in source model selection using the visual task adaptation benchmark (VTAB) [38].”

“As empirical transferability studies [16], [17], [18], [19], [24] incur heavy computational burdens in retraining the transfer learning model on the target training data, a new 2162-237X © 2024 IEEE.”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“Despite being evidently more efficient in computing from practical data than empirical methods, they are also prone to strict data assumptions [7], [20] and insufficient performance [21], [23], while task complexities are similar.”

“Despite being evidently more efficient in computing from practical data than empirical methods, they are also prone to strict data assumptions [7], [20] and insufficient performance [21], [23], while task complexities are similar.”

“Quantitative comparisons with state-of-the-art auxiliary-free transferability metrics including LEEP [21], NCE [20], H-score [7], and LogME [23] are shown in Table II, and visual comparisons are illustrated in Fig.”

“Please refer to [31] and [27] and the official codebase 2 for more details about the URL algorithm.”

“Inspired by recent work [29], we compute the label distance as the Wasserstein distance Wass (α y s , α y t ) .”

“Quantitative comparisons with state-of-the-art auxiliary-free transferability metrics including LEEP [21], NCE [20], H-score [7], and LogME [23] are shown in Table II, and visual comparisons are illustrated in Fig.”

“When multiple source models are accessible, one can transfer them to a target task by merging their inferred features to obtain a fused feature representation [52].”

“The “when to transfer” problem was traditionally studied theoretically through the derivation of generalization bounds of transfer learning across tasks [8], [9] and across domains (also known as the domain adaptation problem) [10], [11], [12], [13], [14], [15].”

“Solving the OT problem by Sinkhorn algorithm with ϵ accuracy has complexity O ( mnk ) = O ( 2 mn ∥ c ∥ 2 /(λϵ)) [37], where ∥ c ∥ ∞ = sup ( z s , z t ) ∈ Z 2 c ( z s , z t ) is the maximum cost between source and target sample features and λ is the weighting coefficient of the entropic regularizer.”

“As empirical transferability studies [16], [17], [18], [19], [24] incur heavy computational burdens in retraining the transfer learning model on the target training data, a new 2162-237X © 2024 IEEE.”

“Please refer to [31] and [27] and the official codebase 2 for more details about the URL algorithm.”

“It is, therefore, imperative to find efficient transferability metrics that can accurately estimate empirical transferability [170].”

“Prior research on domain adaptation [258] and domain generalization [259] highlights this issue but often lacks comprehensive, practical strategies tailored specifically for deepfake detection.”

“While transfer learning reduces dependence on target domain labels, its reliance on source domain labels limits its effectiveness when domain divergence is substantial [21].”

“Achieving good performances on all given domains thus results in a generalizable model [14].”

“Tan et al. [6] trained multiple expert models to learn the differences between the source and target domains, and used a hybrid strategy to fuse the prediction results.”

“NCE can be computed as follows to measure the transferability between source model θ and target model D 61 .”

“Further efforts, including task transferability-driven regulation frameworks (Tan et al. 2024; Ban et al. 2025) and validation in unstructured task spaces for soft robotics (Zhao et al. 2025), demonstrate progress but remain constrained by reliance on full-space parameter sharing and insufficient…”

“MAML-based approaches learn general hyperparameter initialization strategies, significantly reducing tuning costs for new tasks [20].”

“The strong data heterogeneity among different satellites means that performing point-to-point transfer learning typically fails to provide local models with satisfactory benefit, necessitating the selection of more suitable migration sources [6].”

“Concurrently, discrepancy-based approaches focus on explicitly minimizing statistical divergences, such as optimal transport or maximum mean discrepancy, across distinct domains [42].”