Parameter-Efficient Transfer Learning with Diff Pruning

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325

Efficient Estimation of Word Representations in Vector Space

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778

Deep Contextualized Word Representations

Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929

Distributed Representations of Sentences and DocumentsPreprint

Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 7,905

Overcoming catastrophic forgetting in neural networks

Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899

Visual Prompt Tuning

Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965

Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey

Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387

Parameter-efficient fine-tuning of large-scale pre-trained language models

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00626-4 · Cited by 1,191

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

Ben Zaken, E., Goldberg, Y., & Ravfogel, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.1 · Cited by 765

Bias and Fairness in Large Language Models: A Survey

Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245

AdapterFusion: Non-Destructive Task Composition for Transfer Learning

Pfeiffer, J., Kamath, A., Rücklé, A., Cho, K., & Gurevych, I. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.39 · Cited by 636

Towards a Unified View of Parameter-Efficient Transfer LearningPreprint

He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., & Neubig, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04366 · Cited by 757

Self-supervised speech representation learning: A review

Mohamed, A., Lee, H.-Y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., Kirchhoff, K., Li, S.-W., Livescu, K., Maaløe, L., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3207050 · Cited by 372

VL-ADAPTER: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks

Sung, Y.-L., Cho, J., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00516 · Cited by 329

Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment

Xu, L., Xie, H., Qin, S. J., Tao, X., & Wang, F. L. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3657354 · Cited by 565

Frozen CLIP Models are Efficient Video Learners

Lin, Z., Geng, S., Zhang, R., Gao, P., de Melo, G., Wang, X., Dai, J., Qiao, Y., & Li, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19833-5_23 · Cited by 213

Parameter-efficient fine-tuning in large language models: a survey of methodologies

Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11236-4 · Cited by 186

On the Effectiveness of Parameter-Efficient Fine-Tuning

Fu, Z., Yang, H., So, A. M., Lam, W. P., Bing, L., & Collier, N. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26505 · Cited by 187

3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation

Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.-A., & Dou, Q. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103324 · Cited by 156

Block Pruning For Faster Transformers

Lagunas, F., Charlaix, E., Sanh, V., & Rush, A. M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.829 · Cited by 204

Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning

Xu, R., Luo, F., Zhang, Z., Tan, C., Chang, B., Huang, S., & Huang, F. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.749 · Cited by 147

Efficient Methods for Natural Language Processing: A Survey

Treviso, M., Lee, J.-U., Ji, T., van Aken, B., Cao, Q., Ciosici, M. R., Hassid, M., Heafield, K., Hooker, S., Raffel, C., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00577 · Cited by 156

Vision Transformers are Parameter-Efficient Audio-Visual Learners

Lin, Y.-B., Sung, Y.-L., Lei, J., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00228 · Cited by 120

Survey of Different Large Language Model Architectures: Trends, Benchmarks, and Challenges

Shao, M., Basit, A., Karri, R., & Shafique, M. (2024) · IEEE Access · doi:10.1109/access.2024.3482107 · Cited by 130

Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning

Zhang, Z.-R., Tan, C., Xu, H., Wang, C., Huang, J. G., & Huang, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.107 · Cited by 126

UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

Mao, Y., Mathias, L., Hou, R. L., Almahairi, A., Ma, H., Han, J., Yih, S., & Khabsa, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.433 · Cited by 110

Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation

Xu, Z., Chen, Z., Zhang, Y., Song, Y., Wan, X., & Li, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01605 · Cited by 96

E 2 VPT: An Effective and Efficient Approach for Visual Prompt Tuning

Han, C., Wang, Q., Cui, Y., Cao, Z., Wang, W., Qi, S., & Liu, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01604 · Cited by 123

A Survey on Green Deep LearningPreprint

Xu, J., Zhou, W., Fu, Z., Zhou, H., & Li, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05193 · Cited by 118

Sam-based instance segmentation models for the automation of structural damage detection

Ye, Z., Lovell, L., Faramarzi, A., & Ninić, J. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102826 · Cited by 83

Sparse Low-rank Adaptation of Pre-trained Language Models

Ding, N., Lv, X., Wang, Q., Chen, Y., Zhou, B., Liu, Z., & Sun, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.252 · Cited by 145

Composable Sparse Fine-Tuning for Cross-Lingual Transfer

Ansell, A., Ponti, E. M., Korhonen, A., & Vulić, I. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.125 · Cited by 119

Training Neural Networks with Fixed Sparse MasksPreprint

Sung, Y.-L., Nair, V., & Raffel, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09839 · Cited by 84

StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation

Maharana, A., Hannan, D., & Bansal, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_5 · Cited by 81

A Survey on Model Compression and Acceleration for Pretrained Language Models

Xu, C., & McAuley, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26255 · Cited by 85

Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning

He, H., Cai, J., Zhang, J., Tao, D., & Zhuang, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01086 · Cited by 79

Diversity-Aware Meta Visual Prompting

Huang, Q., Dong, X., Chen, D., Zhang, W., Wang, F., Hua, G., & Yu, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01047 · Cited by 76

Generative AI model privacy: a survey

Liu, Y., Huang, J., Li, Y., Wang, D., & Xiao, B. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11024-6 · Cited by 74

DePT: Decoupled Prompt Tuning

Zhang, J., Wu, S., Gao, L., Shen, H. T., & Song, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01228 · Cited by 72

Revisiting Parameter-Efficient Tuning: Are We Really There Yet?

Chen, G., Liu, F., Meng, Z., & Liang, S. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.168 · Cited by 60

The Efficiency MisnomerPreprint

Dehghani, M., Arnab, A., Beyer, L., Vaswani, A., & Tay, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12894 · Cited by 106

Dual-Path Adaptation from Image to Video Transformers

Park, J., Lee, J., & Sohn, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00219 · Cited by 64

An Overview of Language Models: Recent Developments and Outlook

Wei, C., Wang, Y.-C., Wang, B., & Kuo, C. ‐. J. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000010 · Cited by 74

Petals: Collaborative Inference and Fine-tuning of Large Models

Borzunov, A., Baranchuk, D., Dettmers, T., Riabinin, M., Belkada, Y., Chumachenko, A., Samygin, P., & Raffel, C. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · doi:10.18653/v1/2023.acl-demo.54 · Cited by 52

Parameter-Efficient Transfer Learning for Medical Visual Question Answering

Liu, J., Hu, T., Zhang, Y., Yang, F., Hao, J., Lv, J., & Liu, Z. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3311333 · Cited by 52

Parameter-Efficient Fine-Tuning without Introducing New Latency

Liao, B., Meng, Y., & Monz, C. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.233 · Cited by 79

Finicky transfer learning—A method of pruning convolutional neural networks for cracks classification on edge devices

Żarski, M., Wójcik, B., Książek, K., & Miszczak, J. A. (2021) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12755 · Cited by 45

BERT Busters: Outlier Dimensions that Disrupt Transformers

Kovaleva, O., Kulshreshtha, S., Rogers, A., & Rumshisky, A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.300 · Cited by 85

AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks

Fu, C.-L., Chen, Z.-C., Lee, Y.-R., & Lee, H.-Y. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.199 · Cited by 51

Two-stage fine-tuning with ChatGPT data augmentation for learning class-imbalanced data

ValizadehAslani, T., Shi, Y., Wang, J., Ren, P., Zhang, Y., Hu, M., Zhao, L., & Liang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127801 · Cited by 43

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning

Wang, T., Zhou, W., Zeng, Y., & Zhang, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.873 · Cited by 67

A Comprehensive Overview and Analysis of Large Language Models: Trends and Challenges

Mohammed, A., & Kora, R. (2025) · IEEE Access · doi:10.1109/access.2025.3573955 · Cited by 45

CroApp: A CNN-Based Resource Optimization Approach in Edge Computing Environment

Jia, Y., Liu, B., Dou, W., Xu, X., Zhou, X., Qi, L., & Yan, Z. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3154473 · Cited by 37

Distributed Inference and Fine-tuning of Large Language Models Over The Internet

Borzunov, A., Ryabinin, M., Chumachenko, A., Baranchuk, D., Dettmers, T., Belkada, Y., Samygin, P., & Raffel, C. (2023) · arXiv · doi:10.48550/arxiv.2312.08361 · Cited by 35

AutoPEFT : Automatic Configuration Search for Parameter-Efficient Fine-Tuning

Zhou, H., Wan, X., Vulić, I., & Korhonen, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00662 · Cited by 60

LeSAM: Adapt Segment Anything Model for Medical Lesion Segmentation

Gu, Y., Wu, Q., Tang, H. N., Mai, X., Shu, H., Li, B., & Chen, Y. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3406871 · Cited by 37

MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval

Jin, X., Zhang, B., Gong, W., Xu, K., Deng, X., Wang, P., Zhang, Z., Shen, X., & Feng, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02563 · Cited by 58

HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability

Chen, J., Shen, D., Chen, W., & Yang, D. (2021) · doi:10.18653/v1/2021.acl-long.338 · Cited by 35

Gradient-based Parameter Selection for Efficient Fine-Tuning

Zhang, Z., Zhang, Q., Gao, Z., Zhang, R., Shutova, E., Zhou, S., & Zhang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02699 · Cited by 43

Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuningPreprint

Xu, R., Luo, F., Zhang, Z., Tan, C., Chang, B., Huang, S., & Huang, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05687 · Cited by 67

A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models

Guo, C., Cheng, F., Du, Z., Kiessling, J., Ku, J., Li, S., Li, Z., Ma, M., Molom-Ochir, T., Morris, B., et al. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2024.3476008 · Cited by 32

APrompt: Attention Prompt Tuning for Efficient Adaptation of Pre-trained Language Models

Wang, Q., Mao, Y., Wang, J., Yu, H., Nie, S., Wang, S., Feng, F., Huang, L., Quan, X., Xu, Z., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.567 · Cited by 32

AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentation

Li, C., Sultan, R. I., Khanduri, P., Qiang, Y., Chetty, I. J., & Zhu, D. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00352 · Cited by 43

XPrompt: Exploring the Extreme of Prompt Tuning

Ma, F., Zhang, C., Ren, L., Wang, J., Wang, Q., Wu, W., Quan, X., & Song, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.758 · Cited by 45

Efficient LLMs Training and Inference: An Introduction

Li, R., Fu, D., Shi, C., Huang, Z., & Lu, G. (2024) · IEEE Access · doi:10.1109/access.2024.3501358 · Cited by 36

BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting

Yong, Z. X., Schoelkopf, H., Muennighoff, N., Aji, A. F., Adelani, D. I., Almubarak, K., Bari, M. S., Sutawika, L., Kasai, J., Baruwa, A., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.653 · Cited by 30

Prompt tuning for parameter-efficient medical image segmentation

Fischer, M., Bartler, A., & Yang, B. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103024 · Cited by 38

Hydra: Multi-head low-rank adaptation for parameter efficient fine-tuning

Kim, S., Yang, H.-M., Kim, Y., Hong, Y., & Park, E. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106414 · Cited by 36

Learning Expressive Prompting With Residuals for Vision Transformers

Das, R., Dukler, Y., Ravichandran, A., & Swaminathan, A. D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00328 · Cited by 26

Event-Adapted Video Super-Resolution

Xiao, Z., Kai, D., Zhang, Y., Zha, Z.-J., Sun, X., & Xiong, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72946-1_13 · Cited by 29

FLASC: Federated LoRA with Sparse Communication

Kuo, K., Raje, A., Rajesh, K., & Smith, V. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813151 · Cited by 47

Efficient Test Time Adapter Ensembling for Low-resource Language Varieties

Wang, X., Tsvetkov, Y., Ruder, S., & Neubig, G. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.63 · Cited by 30

Universal Deep Image Compression via Content-Adaptive Optimization with Adapters

Tsubota, K., Akutsu, H., & Aizawa, K. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00256 · Cited by 26

MixPAVE: Mix-Prompt Tuning for Few-shot Product Attribute Value Extraction

Yang, L., Wang, Q., Wang, J., Quan, X., Feng, F., Chen, Y., Khabsa, M., Wang, S., Xu, Z., & Liu, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.633 · Cited by 23

Application of transfer learning for biomedical signals: A comprehensive review of the last decade (2014–2024)

Jafari, M., Tao, X., Barua, P. D., Tan, R., & Acharya, U. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102982 · Cited by 27

Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation

Huang, J., Xu, Z., Liu, T., Liu, Y., Han, H., Yuan, K., & Li, X. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i4.32380 · Cited by 33

Towards efficient and accurate CO2 sequestration optimization: Integrating hierarchical spatio-temporal information into deep-learning-based surrogate models

Lin, J., Zhang, K., Zhang, L., Liu, P., Peng, W., Zhang, H., Yan, X., Liu, C., Yang, Y., Sun, H. X., et al. (2023) · Fuel · doi:10.1016/j.fuel.2023.129343 · Cited by 22

Transfer learning for language model adaptation

Saiful, B. M. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/169892 · Cited by 22

Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.06904 · Cited by 20

Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models

Lawton, N., Kumar, A., Thattai, G., Galstyan, A., & Steeg, G. V. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.539 · Cited by 34

Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems

Varshney, N., & Baral, C. R. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.756 · Cited by 39

UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory

Diao, H., Wan, B., Zhang, Y., Jia, X., Lu, H., & Chen, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02714 · Cited by 23

Learning Kernel-Smoothed Machine Translation with Retrieved Examples

Jiang, Q., Wang, M., Cao, J., Cheng, S., Huang, S., & Li, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.579 · Cited by 22

Vietnamese Sentiment Analysis: An Overview and Comparative Study of Fine-tuning Pretrained Language Models

Thin, D. V., Hao, D. N., & Nguyen, N. L.-T. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3589131 · Cited by 21

Structure-Aware Low-Rank Adaptation for Parameter-Efficient Fine-Tuning

Hu, Y., Xie, Y., Wang, T., Chen, M., & Pan, Z. (2023) · Mathematics · doi:10.3390/math11204317 · Cited by 21

MACP: Efficient Model Adaptation for Cooperative Perception

Ma, Y., Lu, J., Cui, C., Zhao, S., Cao, X., Ye, W., & Wang, Z. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00334 · Cited by 22

Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation

Hou, Z., Salazar, J., & Polovets, G. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00517 · Cited by 18

Parameter-Efficient Sparsity for Large Language Models Fine-Tuning

Li, Y., Luo, F., Tan, C., Wang, M., Huang, S., Li, S., & Bai, J. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/586 · Cited by 27

TaskFusion: An Efficient Transfer Learning Architecture with Dual Delta Sparsity for Multi-Task Natural Language Processing

Fan, Z., Zhang, Q., Abillama, P., Shoouri, S., Lee, C. S. G., Blaauw, D. T., Kim, H.-S., & Sylvester, D. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589040 · Cited by 16

Advancing Parameter Efficiency in Fine-tuning via Representation Editing

Wu, M., Liu, W., Wang, X., Li, T., Lv, C., Ling, Z., JianHao, Z., Zhang, C., Zheng, X., & Huang, X. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.726 · Cited by 40

LLMs on a Budget: System-Level Approaches to Power-Efficient and Scalable Fine-Tuning

Gogineni, K., Suvizi, A., & Venkataramani, G. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3580498 · Cited by 14

Beyond Classification: Inferring Function Names in Stripped Binaries via Domain Adapted LLMs

Jiang, L., Jin, X., & Lin, Z. (2025) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2025.240797 · Cited by 16

PEDRO: Parameter-Efficient Fine-Tuning with Prompt DEpenDent Representation MOdification

Xie, T., Li, T., Zhu, W., & Han, W. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6603-4_18 · Cited by 23

MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension

Liu, T., Xu, Z., Hu, Y., Shi, L., Wang, Z., & Yin, Q. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.287 · Cited by 17

Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software

Ran, D., Wu, M. S., Yang, W. M., & Xie, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3719005 · Cited by 15

Experimental Evaluation of Parameter-Efficient Fine-Tuning for Software Engineering Tasks

Zou, W., Shen, Z., Li, Q., Ge, J., Li, C., Chen, X., Shen, X., Huang, L., & Luo, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3722107 · Cited by 16

Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling

Shim, K., Choi, I., Sung, W., & Choi, J. (2021) · International SoC Design Conference · doi:10.1109/isocc53507.2021.9613933 · Cited by 19

Personalized LoRA for Human-Centered Text Understanding

Zhang, Y., Wang, J., Yu, L., Xu, D., & Zhang, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29931 · Cited by 16

Facial Affective Behavior Analysis with Instruction Tuning

Li, Y., Dao, A., Bao, W., Tan, Z., Chen, T., Liu, H., & Kong, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_10 · Cited by 18

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

Wang, Z., Chen, D., Dai, D., Xu, R., Li, Z., & Wu, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.46 · Cited by 27

Enhancing Large Language Models for Specialized Domains: A Two-Stage Framework with Parameter-Sensitive LoRA Fine-Tuning and Chain-of-Thought RAG

He, Y., Zhu, X., Li, D., & Wang, H. (2025) · Electronics · doi:10.3390/electronics14101961 · Cited by 16

Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks

Hauzenberger, L., Masoudian, S., Kumar, D., Schedl, M., & Rekabsaz, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.386 · Cited by 16

Non-Parametric Domain Adaptation for End-to-End Speech Translation

Du, Y., Wang, W., Zhang, Z., Chen, B., Xu, T. W., Xie, J., & Chen, E. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.21 · Cited by 10

SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts

Choi, J., Kim, J., Park, J.-H., Mok, W.-L., & Lee, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.884 · Cited by 12

Shannon Strikes Again! Entropy-based Pruning in Deep Neural Networks for Transfer Learning under Extreme Memory and Computation Budgets

Spadaro, G., Renzulli, R., Bragagnolo, A., Giraldo, J. H., Fiandrotti, A., Grangetto, M., & Tartaglione, E. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00165 · Cited by 10

Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfer

Du, G., Fang, Z., Li, J., Li, J., Jiang, R., Yu, S., Guo, Y., Chen, Y., Goh, S. K., Tang, H.-K., et al. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1570 · Cited by 4

DimA: A Parameter-efficient Fine-tuning Method with Knowledge Transfer Based on Transformer

Zhang, W., Huang, M., Song, Z., & Miao, Q. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/34asv3xtjq9h

Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning

Poth, C., Sterz, H., Paul, I., Purkayastha, S., Engländer, L., Imhof, T., Vuli, I., Ruder, S., Gurevych, I., & Pfeiffer, J. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.13 · Cited by 39

Efficient transfer learning with pretrained language models

Zhao, M. (2022) · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · doi:10.5282/edoc.30059

An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models

Chronopoulou, A., Baziotis, C., & Potamianos, A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1213 · Cited by 108