The DOI matches its DataCite record; 47 of the 49 references it cites were found in a registry or an index, and 1 are not checked yet.
The Power of Scale for Parameter-Efficient Prompt Tuning
Open accessAbstract
In this work, we explore "prompt tuning", a simple yet effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signal from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3's "few-shot" learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned). This finding is especially relevant in that large models are costly to share and serve, and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification of the recently proposed "prefix tuning" of Li and Liang (2021), and we provide a comparison to this and other similar approaches. Finally, we show that conditioning a frozen model with soft prompts confers benefits in robustness to domain transfer, as compared to full model tuning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These approaches become inapplicable when there is only API access to the LLM.”
From Large Language Models as Optimizers (2023) · cites this paper as Lester et al., 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2010) · International Conference on Machine Learning · Cited by 13,995
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
seaborn: statistical data visualization
Waskom, M. (2021) · Journal of Open Source Software · doi:10.21105/joss.03021 · Cited by 7,310
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Research works printed but not linked: 8
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Marie-Catherine De Marneff, Mandy Simons, and Ju- dith Tonhauser. 2019. The CommitmentBank: In- vestigating projection in naturally occurring dis- course. Proceedings of Sinn und Bedeutung 23.
- Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recog- nizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1-9. Association for Com- putational Linguistics.
- PMLR.
- Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
- Melissa Roemmele, Cosmin Adrian Bejan, and An- drew S Gordon. 2011. Choice of plausible alterna- tives: An evaluation of commonsense causal reason- ing. In 2011 AAAI Spring Symposium Series.
- Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4596-4604.
- PMLR.
Cited by
Sorted by citations
Items 201–300 of 1,009
DeepStruct: Pretraining of Language Models for Structure Prediction
Wang, C., Liu, X., Chen, Z., Hong, H., Tang, J., & Song, D. X. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.67 · Cited by 109
Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Models
van Sonsbeek, T., Derakhshani, M. M., Najdenkoska, I., Snoek, C. G. M., & Worring, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43904-9_70 · Cited by 102
MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Zhang, Y., Huang, D., Liu, B., Tang, S., Lu, Y., Chen, L., Bai, L., Chu, Q., Yu, N., & Ouyang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28567 · Cited by 128
Combined scaling for zero-shot transfer learning
Pham, H., Dai, Z., Ghiasi, G., Kawaguchi, K., Liu, H., Yu, A. W., Yu, J., Chen, Y.-T., Luong, M.-T., Wu, Y., et al. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126658 · Cited by 85
Prompt-Learning for Short Text Classification
Zhu, Y. Q., Wang, Y., Qiang, J., & Wu, X. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3332787 · Cited by 84
Large models for intelligent transportation systems and autonomous vehicles: A survey
Gan, L., Chu, W., Li, G., Tang, X., & Li, K. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102786 · Cited by 81
A Survey on Efficient Training of Transformers
Zhuang, B., Liu, J., Pan, Z., He, H., Weng, Y., & Shen, C. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/764 · Cited by 78
Iteratively Prompt Pre-trained Language Models for Chain of Thought
Wang, B., Deng, X. B., & Sun, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.174 · Cited by 123
A Survey of Robot Intelligence with Large Language Models
Jeong, H., Lee, H., Kim, C., & Shin, S. (2024) · Applied Sciences · doi:10.3390/app14198868 · Cited by 89
Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities
Abdali, S., Shaham, S., & Krishnamachari, B. (2024) · ACM Computing Surveys · doi:10.1145/3697349 · Cited by 107
CPM-2: Large-scale cost-effective pre-trained language models
Zhang, Z., Gu, Y., Han, X., Chen, S., Xiao, C., Sun, Z., Yao, Y., Qi, F., Guan, J., Ke, P., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.12.003 · Cited by 92
Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance
Lin, L., Fan, H., Zhang, Z., Wang, Y., Xu, Y., & Ling, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_17 · Cited by 86
Meta-learning via Language Model In-context Tuning
Chen, Y., Zhong, R., Zha, S., Karypis, G., & He, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.53 · Cited by 143
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
Asai, A., Salehi, M., Peters, M. E., & Hajishirzi, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.446 · Cited by 103
VoP: Text-Video Co-Operative Prompt Tuning for Cross-Modal Retrieval
Huang, S., Gong, B., Pan, Y., Jiang, J., Lv, Y., Li, Y., & Wang, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00635 · Cited by 78
Cheng, P., Yang, X., Smith, K. E., Yu, Z., Chen, A., Bian, J., & Wu, Y. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104630 · Cited by 85
Orthogonal Subspace Learning for Language Model Continual Learning
Wang, X., Chen, T., Ge, Q., Xia, H., Bao, R., Zheng, R., Zhang, Q., Gui, T., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.715 · Cited by 79
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Wang, Y., Agarwal, S., Mukherjee, S., Liu, X., Gao, J., Awadallah, A. H., & Gao, J. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.388 · Cited by 77
PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks
Wang, Y., Xu, C., Sun, Q., Hu, H., Tao, C., Geng, X., & Jiang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.292 · Cited by 77
Adversarial Soft Prompt Tuning for Cross-Domain Sentiment Analysis
Wu, H., & Shi, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.174 · Cited by 76
Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models
Ma, C., Liu, Y., Deng, J., Xie, L., Dong, W., & Xu, C. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3245584 · Cited by 84
Consistency Analysis of ChatGPT
Jang, M., & Lukasiewicz, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.991 · Cited by 117
A Survey on Employing Large Language Models for Text-to-SQL Tasks
Shi, L., Tang, Z., Zhang, N., Zhang, X., & Yang, Z. (2025) · ACM Computing Surveys · doi:10.1145/3737873 · Cited by 156
One for All: A Unified Generative Framework for Image Emotion Classification
Shi, G., Deng, S., Wang, B., Feng, C., Zhuang, Y., & Wang, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3341840 · Cited by 73
Training Neural Networks with Fixed Sparse MasksPreprint
Sung, Y.-L., Nair, V., & Raffel, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09839 · Cited by 84
Generative Artificial Intelligence for Software Engineering - a Research Agenda
NguyenDuc, A., Cabrero-Daniel, B., Arora, C., Przybylek, A., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K.-K., et al. (2023) · doi:10.2139/ssrn.4622517 · Cited by 71
Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets
Chen, H., Tao, R., Zhang, H., Wang, Y., Li, X., Ye, W., Wang, J., Hu, G., & Savvides, M. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00162 · Cited by 105
Emergent social conventions and collective bias in LLM populations
Ashery, A. F., Aiello, L. M., & Baronchelli, A. (2025) · Science Advances · doi:10.1126/sciadv.adu9368 · Cited by 81
StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation
Maharana, A., Hannan, D., & Bansal, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_5 · Cited by 81
Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Prompting
Wen, Z., & Fang, Y. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591641 · Cited by 82
Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models
Weyssow, M., Zhou, X., Kim, K., Lo, D., & Sahraoui, H. A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3714461 · Cited by 158
E 2 VPT: An Effective and Efficient Approach for Visual Prompt Tuning
Han, C., Wang, Q., Cui, Y., Cao, Z., Wang, W., Qi, S., & Liu, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01604 · Cited by 123
Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning
Liu, X.-Y., Zhu, R., Zha, D., Gao, J., Zhong, S., White, M., & Qiu, M. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3682068 · Cited by 100
Read-only Prompt Optimization for Vision-Language Few-shot Learning
Lee, D., Song, S., Suh, J., Choi, J., Lee, S., & Kim, H. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00135 · Cited by 73
Current and future state of evaluation of large language models for medical summarization tasks
Croxford, E., Gao, Y., Pellegrino, N., Wong, K. K. Y., Wills, G. J., First, E., Liao, F., Goswami, C., Patterson, B. W., & Afshar, M. (2025) · npj Health Systems · doi:10.1038/s44401-024-00011-2 · Cited by 81
Diversity-Aware Meta Visual Prompting
Huang, Q., Dong, X., Chen, D., Zhang, W., Wang, F., Hua, G., & Yu, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01047 · Cited by 76
CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognition
Hegde, D., Valanarasu, J. M. J., & Patel, V. M. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00217 · Cited by 87
Multitask Vision-Language Prompt Tuning
Shen, S., Yang, S., Zhang, T., Zhai, B., Gonzalez, J. E., Keutzer, K., & Darrell, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00556 · Cited by 85
Virtual prompt pre-training for prototype-based few-shot relation extraction
He, K., Huang, Y., Mao, R., Gong, T., Li, C., & Cambria, E. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118927 · Cited by 65
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
He, H., Cai, J., Zhang, J., Tao, D., & Zhuang, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01086 · Cited by 79
Learning Domain Invariant Prompt for Vision-Language Models
Zhao, C., Wang, Y., Jiang, X., Shen, Y., Song, K., Li, D., & Miao, D. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3362062 · Cited by 81
Tanaka, Y., Nakata, T., Aiga, K., Etani, T., Muramatsu, R., Katagiri, S., Kawai, H., Higashino, F., Enomoto, M., Noda, M., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000433 · Cited by 65
Graph Neural Prompting with Large Language Models
Tian, Y., Song, H., Wang, Z., Wang, H., Hu, Z., Wang, F., Chawla, N. V., & Xu, P. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29875 · Cited by 105
Understanding LLMs: A comprehensive overview from training to inference
Liu, Y., He, H., Han, T., Zhang, X., Liu, M., Tian, J., Zhang, Y., Wang, J., Gao, X., Zhong, T., et al. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129190 · Cited by 81
True Few-Shot Learning with Prompts—A Real-World Perspective
Schick, T., & Schütze, H. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00485 · Cited by 81
CIRCLE: continual repair across programming languages
Yuan, W., Zhang, Q., He, T., Fang, C., Nguyen, Q. V. H., Hao, X., & Yin, H. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534219 · Cited by 82
Schramowski, P., Tauchmann, C., & Kersting, K. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533192 · Cited by 86
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
Shum, K., Diao, S., & Zhang, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.811 · Cited by 79
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization
Kim, H., Hessel, J., Jiang, L., West, P., Lu, X., Yu, Y., Zhou, P., Bras, R., Alikhani, M., Kim, G., et al. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.799 · Cited by 63
Open challenges and opportunities in federated foundation models towards biomedical healthcare
Li, X., Peng, L., Wang, Y., & Zhang, W. (2025) · BioData Mining · doi:10.1186/s13040-024-00414-9 · Cited by 84
Combining Fine-Tuning and LLM-Based Agents for Intuitive Smart Contract Auditing with Justifications
Ma, W., Wu, D., Sun, Y., Wang, T., Liu, S., Zhang, J., Xue, Y., & Liu, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00027 · Cited by 127
Cell2Sentence: Teaching Large Language Models the Language of BiologyPreprint
Levine, D. M., Rizvi, S. A., Lévy, S., Pallikkavaliyaveetil, N., Zhang, D., Chen, X., Ghadermarzi, S., Wu, R., Zheng, Z., Vrkic, I., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.09.11.557287 · Cited by 68
A review of LLMs and their applications in the architecture, engineering and construction industry
Kampelopoulos, D., Tsanousa, A., Vrochidis, S., & Kompatsiaris, I. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11241-7 · Cited by 69
PromptNER: Prompt Locating and Typing for Named Entity Recognition
Shen, Y., Tan, Z., Wu, S., Zhang, W., Zhang, R., Xi, Y., Lü, W., & Zhuang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.698 · Cited by 82
Wang, Y., Cheng, J., Chen, Y., Shao, S., Zhu, L., Wu, Z., Liu, T., & Zhu, H. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3306105 · Cited by 66
Sparse Low-rank Adaptation of Pre-trained Language Models
Ding, N., Lv, X., Wang, Q., Chen, Y., Zhou, B., Liu, Z., & Sun, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.252 · Cited by 145
Generating Instance-level Prompts for Rehearsal-free Continual Learning
Jung, D., Han, D., Bang, J., & Song, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01088 · Cited by 69
Harnessing domain insights: A prompt knowledge tuning method for aspect-based sentiment analysis
Sun, X., Zhang, K., Liu, Q., Bao, M., & Chen, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111975 · Cited by 66
On Transferability of Prompt Tuning for Natural Language Processing
Su, Y., Wang, X., Qin, Y., Chan, C.-M., Lin, Y., Wang, H., Wen, K., Liu, Z., Li, P., Li, J., et al. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.290 · Cited by 98
Chen, C., Wang, Y., Sun, A., Li, B., & Lam, K. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.729 · Cited by 83
Rao, H., & Miao, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02118 · Cited by 63
Few-shot adaptation of multi-modal foundation models: a survey
Liu, F., Zhang, T., Dai, W., Zhang, C., Cai, W., Zhou, X., & Chen, D. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10915-y · Cited by 75
Continual Learning With Knowledge Distillation: A Survey
Li, S., Su, T., Zhang, X.-Y., & Wang, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476068 · Cited by 71
Vision-Language Models for Edge Networks: A Comprehensive Survey
Sharshar, A., Khan, L. U., Ullah, W., & Guizani, M. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3579032 · Cited by 92
Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-training
Hardalov, M., Arora, A., Nakov, P., & Augenstein, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21318 · Cited by 66
KEPT: Knowledge Enhanced Prompt Tuning for event causality identification
Liu, J., Zhang, Z., Guo, Z. M., Jin, L., Li, X., Wei, K., & Sun, X. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110064 · Cited by 60
Few-shot Text-to-SQL Translation using Structure and Content Prompt Learning
Gu, Z., Fan, J., Tang, N., Cao, L., Jia, B., Madden, S. R., & Du, X. (2023) · Proceedings of the ACM on Management of Data · doi:10.1145/3589292 · Cited by 60
Revisiting Parameter-Efficient Tuning: Are We Really There Yet?
Chen, G., Liu, F., Meng, Z., & Liang, S. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.168 · Cited by 60
Efficient Multimodal Fusion via Interactive Prompting
Li, Y., Quan, R., Zhu, L., & Yang, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00256 · Cited by 60
Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning
Lu, X., Guo, S., Liu, Z., & Guo, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02256 · Cited by 65
LLMaAA: Making Large Language Models as Active Annotators
Zhang, R., Li, Y., Ma, Y., Zhou, M. Q., & Zou, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.872 · Cited by 114
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60
Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation
Yang, F.-E., Wang, C.-Y., & Wang, Y.-C. F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01755 · Cited by 68
Fairness in Large Language Models: A Taxonomic Survey
Chu, Z., Wang, Z., & Zhang, W. (2024) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3682112.3682117 · Cited by 106
Foundation models and intelligent decision-making: Progress, challenges, and perspectives
Huang, J., Xu, Y., Wang, Q., Wang, Q., Liang, X., Wang, F., Zhang, Z., Wei, W., Zhang, B., Huang, L., et al. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100948 · Cited by 87
Evaluating the representational power of pre-trained DNA language models for regulatory genomics
Tang, Z., Somia, N., Yu, Y., & Koo, P. K. (2025) · Genome biology · doi:10.1186/s13059-025-03674-8 · Cited by 71
Generative Artificial Intelligence for Software Engineering—A Research Agenda
Nguyen‐Duc, A., Cabrero‐Daniel, B., Przybylek, A., Arora, C., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K., et al. (2025) · Software: Practice and Experience · doi:10.1002/spe.70005 · Cited by 60
Understanding and Improving Visual Prompting: A Label-Mapping Perspective
Chen, A., Yao, Y., Chen, P., Zhang, Y., & Liu, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01834 · Cited by 69
EHR-KnowGen: Knowledge-enhanced multimodal learning for disease diagnosis generation
Niu, S., Ma, J., Bai, L., Wang, Z., Guo, L., & Yang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102069 · Cited by 59
Label prompt for multi-label text classification
Song, R., Liu, Z., Chen, X., An, H., Zhang, Z., Wang, X., & Xu, H. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03896-4 · Cited by 64
Multimodal LLMs for Health Grounded in Individual-Specific Data
Belyaeva, A., Cosentino, J., Hormozdiari, F., Eswaran, K., Shetty, S., Corrado, G. S., Carroll, A. W., McLean, C. Y., & Furlotte, N. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47679-2_7 · Cited by 74
Xie, E., Yao, L., Shi, H., Liu, Z., Zhou, D., Liu, Z., Li, J., & Li, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00390 · Cited by 97
Towards Large-Scale 3D Representation Learning with Multi-Dataset Point Prompt Training
Wu, X., Tian, Z., Wen, X., Peng, B., Liu, X., Yu, K., & Zhao, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01849 · Cited by 72
On the Effectiveness of Large Language Models in Domain-Specific Code Generation
Gu, X., Chen, M., Lin, Y., Hu, Y., Zhang, H., Wan, C., Wei, Z., Xu, Y., & Wang, J. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697012 · Cited by 92
Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Survey
Vu, T.-H., Jagatheesaperumal, S. K., Nguyen, M.-D., Huynh, N. V., Kim, S., & Pham, Q. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3487627 · Cited by 78
Ladder Fine-tuning Approach for SAM Integrating Complementary Network
Chai, S., Jain, R. K., Teng, S., Liu, J., Li, Y., Tateyama, T., & Chen, Y. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.452 · Cited by 77
Mayer, C., Ludwig, S., & Brandt, S. (2022) · Journal of Research on Technology in Education · doi:10.1080/15391523.2022.2142872 · Cited by 72
Zero-Shot Rumor Detection with Propagation Structure via Prompt Learning
Lin, H., Yi, P., Ma, J., Jiang, H., Luo, Z., Shi, S., & Liu, R. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25651 · Cited by 59
Large language model for table processing: a survey
Lu, W., Zhang, J., Fan, J., Fu, Z., Chen, Y., & Du, X. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-024-40763-6 · Cited by 107
Notice of Removal: Semantic Prompt for Few-Shot Image Recognition
Chen, W., Si, C., Zhang, Z., Wang, L., Wang, Z., & Tan, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02258 · Cited by 59
Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models
Zhao, S., Wen, J., Luu, A. N., Zhao, J., & Fu, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.757 · Cited by 96
Bi-directional Training for Composed Image Retrieval via Text Prompt Learning
Liu, Z., Sun, W., Hong, Y., Teney, D., & Gould, S. J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00565 · Cited by 57
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 133
End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames
Liu, S., Zhang, C.-L., Zhao, C., & Ghanem, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01759 · Cited by 71
Efficient utilization of pre-trained models: A review of sentiment analysis via prompt learning
Bu, K., Liu, Y., & Ju, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111148 · Cited by 61
Generative AI model privacy: a survey
Liu, Y., Huang, J., Li, Y., Wang, D., & Xiao, B. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11024-6 · Cited by 74
PLeak: Prompt Leaking Attacks against Large Language Model Applications
Hui, B., Yuan, H., Gong, N., Burlina, P., & Cao, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3670370 · Cited by 55
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains
Ben-David, E., Oved, N., & Reichart, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00468 · Cited by 54
Chen, Y., Weng, Y.-C., Kao, C.-H., Chien, C., Chiu, W.-C., & Peng, W.-H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02129 · Cited by 67
Cheng, Y., & Liu, D. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3420265 · Cited by 61
Metadata
- Authors
- Lester, B., Al-Rfou, R., & Constant, N.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 3045-3059
- arXiv
- 2104.08691
- OpenAlex
- W3152956381
- Cited by
- 3,092
- References
- 49
- Citations by year
- 2026: 512 · 2025: 889 · 2024: 754 · 2023: 649 · 2022: 246 · 2021: 40
- FWCI
- 195.07
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- prompt tuning, soft prompts, parameter-efficient fine-tuning, frozen language models, few-shot learning, model size, prefix tuning, T5, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
