The DOI matches its DataCite record; 47 of the 49 references it cites were found in a registry or an index, and 1 are not checked yet.
The Power of Scale for Parameter-Efficient Prompt Tuning
Open accessAbstract
In this work, we explore "prompt tuning", a simple yet effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signal from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3's "few-shot" learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned). This finding is especially relevant in that large models are costly to share and serve, and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification of the recently proposed "prefix tuning" of Li and Liang (2021), and we provide a comparison to this and other similar approaches. Finally, we show that conditioning a frozen model with soft prompts confers benefits in robustness to domain transfer, as compared to full model tuning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These approaches become inapplicable when there is only API access to the LLM.”
From Large Language Models as Optimizers (2023) · cites this paper as Lester et al., 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2010) · International Conference on Machine Learning · Cited by 13,995
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
seaborn: statistical data visualization
Waskom, M. (2021) · Journal of Open Source Software · doi:10.21105/joss.03021 · Cited by 7,310
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Research works printed but not linked: 8
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Marie-Catherine De Marneff, Mandy Simons, and Ju- dith Tonhauser. 2019. The CommitmentBank: In- vestigating projection in naturally occurring dis- course. Proceedings of Sinn und Bedeutung 23.
- Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recog- nizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1-9. Association for Com- putational Linguistics.
- PMLR.
- Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Re- search, 21(140):1-67.
- Melissa Roemmele, Cosmin Adrian Bejan, and An- drew S Gordon. 2011. Choice of plausible alterna- tives: An evaluation of commonsense causal reason- ing. In 2011 AAAI Spring Symposium Series.
- Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4596-4604.
- PMLR.
Cited by
Sorted by citations
Items 101–200 of 1,009
PromptMaker: Prompt-based Prototyping with Large Language Models
Jiang, E., Olson, K., Toh, E., Molina, A., Donsbach, A., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503564 · Cited by 157
Large Language Models as Optimizers
Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., & Chen, X. (2023) · arXiv · doi:10.48550/arxiv.2309.03409 · Cited by 151
No more fine-tuning? an experimental evaluation of prompt tuning in code intelligence
Wang, C., Yang, Y., Gao, C., Peng, Y. F., Zhang, H., & Lyu, M. R.-T. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549113 · Cited by 161
Visual Adversarial Examples Jailbreak Aligned Large Language Models
Qi, X., Huang, K., Panda, A., Henderson, P., Wang, M., & Mittal, P. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30150 · Cited by 150
Prompt Engineering Paradigms for Medical Applications: Scoping Review
Zaghir, J., Naguib, M., Bjelogrlic, M., Névéol, A., Tannier, X., & Lovis, C. (2024) · Journal of Medical Internet Research · doi:10.2196/60501 · Cited by 143
Is GPT-3 a Good Data Annotator?
Ding, B., Qin, C., Liu, L., Chia, Y. K., Li, B., Joty, S., & Bing, L. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.626 · Cited by 149
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey
Ling, C., Zhao, X., Lu, J., Deng, C., Zheng, C., Wang, J., Chowdhury, T., Li, Y., Cui, H., Zhang, X., et al. (2025) · ACM Computing Surveys · doi:10.1145/3764579 · Cited by 385
Large Language Models for Code: Security Hardening and Adversarial Testing
He, J., & Vechev, M. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3623175 · Cited by 229
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
Saito, K., Sohn, K., Zhang, X., Li, C.-L., Lee, C.-Y., Saenko, K., & Pfister, T. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01850 · Cited by 145
Waqas, A., Bui, M. M., Glassy, E. F., Naqa, I. E., Borkowski, P., Borkowski, A., & Rasool, G. (2023) · Laboratory Investigation · doi:10.1016/j.labinv.2023.100255 · Cited by 158
PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation
Jin, H., Che, H., Lin, Y., & Chen, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28038 · Cited by 181
Multimodal Prompting with Missing Modalities for Visual Recognition
Lee, Y.-L., Tsai, Y.-H., Chiu, W.-C., & Lee, C.-Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01435 · Cited by 156
Prompt-learning for Fine-grained Entity Typing
Ding, N., Chen, Y., Han, X., Xu, G., Wang, X., Xie, P., Zheng, H., Liu, Z., Li, J., & Kim, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.512 · Cited by 166
A survey on LoRA of large language models
Mao, Y., Ge, Y., Fan, Y., Xu, W., Mi, Y., Hu, Z., & Gao, Y. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40663-9 · Cited by 137
FD-LLM: Large language model for fault diagnosis of complex equipment
Lin, L., Zhang, S., Fu, S., & Liu, Y. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103208 · Cited by 153
Xu, L., Xie, H., Qin, S. J., Tao, X., & Wang, F. L. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3657354 · Cited by 565
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Logan IV, R., Balazevic, I., Wallace, E., Petroni, F., Singh, S., & Riedel, S. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.222 · Cited by 133
Noisy Channel Language Model Prompting for Few-Shot Text Classification
Min, S., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.365 · Cited by 181
EDICT: Exact Diffusion Inversion via Coupled Transformations
Wallace, B., Gokul, A., & Naik, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02158 · Cited by 179
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails
Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C., & Cohen, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2023.emnlp-demo.40 · Cited by 220
Valipour, M., Rezagholizadeh, M., Kobyzev, I., & Ghodsi, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.239 · Cited by 304
PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
Feng, Y., Wang, X., Wong, K. K., Wang, S., Lu, Y., Zhu, M., Wang, B., & Chen, W. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327168 · Cited by 124
RingMo-SAM: A Foundation Model for Segment Anything in Multimodal Remote-Sensing Images
Yan, Z., Li, J., Li, X., Zhou, R., Zhang, W., Feng, Y., Diao, W., Fu, K., & Sun, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3332219 · Cited by 150
Kuang, W., Qian, B., Li, Z., Chen, D., Gao, D., Pan, X., Xie, Y., Li, Y., Ding, B., & Zhou, J. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671573 · Cited by 142
SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking
Hou, X., Xing, J., Qian, Y., Guo, Y., Xin, S., Chen, J., Tang, K., Wang, M., Jiang, Z., Liu, L., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02507 · Cited by 143
Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection
Liu, H., Tan, Z., Tan, C., Wei, Y., Wang, J., & Zhao, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01024 · Cited by 176
Reframing Instructional Prompts to GPTk’s Language
Mishra, S. R., Khashabi, D., Baral, C. R., Choi, Y., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.50 · Cited by 140
LAnoBERT: System log anomaly detection based on BERT masked language model
Lee, Y., Kim, J., & Kang, P. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110689 · Cited by 159
HIPTrack: Visual Tracking with Historical Prompts
Cai, W., Liu, Q., & Wang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01822 · Cited by 146
Cui, C., Ma, Y., Cao, X., Ye, W., & Wang, Z. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00101 · Cited by 156
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting
Wasim, S. T., Naseer, M., Khan, S., Khan, F. S., & Shah, M. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02206 · Cited by 140
LLMs4OL: Large Language Models for Ontology Learning
Giglou, H. B., D’Souza, J., & Auer, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47240-4_22 · Cited by 212
SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model
Zhang, G., Wang, L., Kang, G., Chen, L., & Wei, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01754 · Cited by 156
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
Baek, J., Aji, A. F., & Saffari, A. (2023) · NLRSE · doi:10.18653/v1/2023.nlrse-1.7 · Cited by 274
Wang, D., Feng, L., Ye, J., Zou, J., & Zheng, Y. (2023) · MedComm – Future Medicine · doi:10.1002/mef2.43 · Cited by 127
Task Residual for Tuning Vision-Language Models
Yu, T., Lu, Z., Jin, X., Chen, Z., & Wang, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01049 · Cited by 143
RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions
Wang, Y., Shen, S., & Lim, B. Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581402 · Cited by 120
Large language models: a survey of their development, capabilities, and applications
Yadagiri, A., & Pakray, P. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02310-4 · Cited by 155
Lu, J., Yu, L., Xiao-jia, L., Li, Y., & Zuo, C. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00026 · Cited by 172
Wei, Z., Chen, L., Jin, Y., Ma, X., Liu, T., Ling, P., Wang, B., Chen, H., & Zheng, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02704 · Cited by 133
Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles
Cui, C., Ma, Y., Cao, X., Ye, W., & Wang, Z. (2024) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2024.3381793 · Cited by 137
State of What Art? A Call for Multi-Prompt LLM Evaluation
Mizrahi, M., Kaplan, G., Malkin, D., Dror, R., Shahaf, D., & Stanovsky, G. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00681 · Cited by 113
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor
Honovich, O., Scialom, T., Levy, O., & Schick, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.806 · Cited by 324
An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels
Sorensen, T., Robinson, J., Rytting, C., Shaw, A., Rogers, K., Delorey, A., Khalil, M., Fulda, N., & Wingate, D. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.60 · Cited by 113
Vision Transformers are Parameter-Efficient Audio-Visual Learners
Lin, Y.-B., Sung, Y.-L., Lei, J., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00228 · Cited by 120
Demystifying Prompts in Language Models via Perplexity Estimation
Gonen, H., Iyer, S., Blevins, T., Smith, N., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.679 · Cited by 109
Efficient Methods for Natural Language Processing: A Survey
Treviso, M., Lee, J.-U., Ji, T., van Aken, B., Cao, Q., Ciosici, M. R., Hassid, M., Heafield, K., Hooker, S., Raffel, C., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00577 · Cited by 156
From Large Language Models to Large Multimodal Models: A Literature Review
Huang, D., Yan, C., Li, Q., & Peng, X. (2024) · Applied Sciences · doi:10.3390/app14125068 · Cited by 125
Recent Advances in Large Language Models for Healthcare
Nassiri, K., & Akhloufi, M. A. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4020062 · Cited by 108
A Unified Continual Learning Framework with General Parameter-Efficient Tuning
Gao, Q., Zhao, C., Sun, Y., Xi, T., Zhang, G., Ghanem, B., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01055 · Cited by 104
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105
Dynamic Prefix-Tuning for Generative Template-based Event Extraction
Liu, X., Huang, H., Shi, G., & Wang, B. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.358 · Cited by 113
Materials science in the era of large language models: a perspective
Lei, G., Docherty, R., & Cooper, S. J. (2024) · Digital Discovery · doi:10.1039/d4dd00074a · Cited by 130
BB-GeoGPT: A framework for learning a large language model for geographic information science
Zhang, Y., Wang, Z., He, Z., Li, J., Mai, G., Lin, J., Cheng, W., & Yu, W. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103808 · Cited by 109
Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections
Zhong, R., Lee, K., Zhang, Z., & Klein, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.244 · Cited by 152
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Bačkurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2024) · Journal of Privacy and Confidentiality · doi:10.29012/jpc.880 · Cited by 307
Active Example Selection for In-Context Learning
Zhang, Y., Shi, F., & Tan, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.622 · Cited by 180
AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580907 · Cited by 98
Large Language Models Can Be Strong Differentially Private LearnersPreprint
Li, X., Tramèr, F., Liang, P., & Hashimoto, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05679 · Cited by 291
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
Laskar, M. T. R., Bari, M. S., Rahman, M., Bhuiyan, M. A. H., Joty, S., & Huang, J. X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.29 · Cited by 160
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning
Zhang, Z.-R., Tan, C., Xu, H., Wang, C., Huang, J. G., & Huang, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.107 · Cited by 126
Herrmann‐Werner, A., Wietek, T. F., Holderried, F., Herschbach, L., Griewatz, J., Masters, K., Zipfel, S., & Mahling, M. (2024) · Journal of Medical Internet Research · doi:10.2196/52113 · Cited by 109
Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models
Louis, A., DIJCK, G. V., & Spanakis, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30232 · Cited by 132
Zhang, Z., Yang, Y., Dai, Y., Wang, Q., Yu, Y., Qu, L., & Xu, Z. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.632 · Cited by 121
Survey of Different Large Language Model Architectures: Trends, Benchmarks, and Challenges
Shao, M., Basit, A., Karri, R., & Shafique, M. (2024) · IEEE Access · doi:10.1109/access.2024.3482107 · Cited by 130
Temporal Adaptive RGBT Tracking with Modality Prompt
Wang, H., Liu, X., Li, Y., Sun, M., Yuan, D., & Liu, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28352 · Cited by 105
Integrating LLMs With ITS: Recent Advances, Potentials, Challenges, and Future Directions
Mahmud, D., Hajmohamed, H., Almentheri, S., Alqaydi, S., Aldhaheri, L., Khalil, R. A., & Saeed, N. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3528116 · Cited by 116
Application of large language models in medicine
Liu, F., Zhou, H., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Hua, Y., Zhou, P., et al. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00279-5 · Cited by 94
Visual Prompt Tuning for Generative Transfer Learning
Sohn, K., Chang, H., Lezama, J., Polanía, L. F., Zhang, H., Hao, Y., Essa, I. A., & Jiang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01900 · Cited by 103
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
Mao, Y., Mathias, L., Hou, R. L., Almahairi, A., Ma, H., Han, J., Yih, S., & Khabsa, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.433 · Cited by 110
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model
Xing, Y., Wu, Q., Cheng, D., Zhang, S., Liang, G., Wang, P., & Zhang, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3291588 · Cited by 129
Parameter-Efficient Transfer Learning for Remote Sensing Image–Text Retrieval
Yuan, Y., Zhan, Y., & Xiong, Z. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3308969 · Cited by 94
Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text
Mihindukulasooriya, N., Tiwari, S. M., Enguix, C. F., & Lata, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47243-5_14 · Cited by 126
The Large Language Models on Biomedical Data Analysis: A Survey
Lan, W., Tang, Z., Liu, M., Chen, Q., Peng, W., Chen, Y. P., & Pan, Y. F. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3530794 · Cited by 100
Improving Passage Retrieval with Zero-Shot Question Generation
Sachan, D., Lewis, M., Joshi, M., Aghajanyan, A., Yih, W.-T., Pineau, J., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.249 · Cited by 90
FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation
Oin, J., Wu, J., Yan, P., Li, M., Yuxi, R., Xiao, X., Wang, Y., Wang, R., Wen, S., Pan, X., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01863 · Cited by 106
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
He, Z., Sun, T., Tang, Q., Wang, K., Huang, X., & Qiu, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.248 · Cited by 206
MMA: Multi-Modal Adapter for Vision-Language Models
Yang, L., Zhang, R., Wang, Y., & Xie, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02249 · Cited by 108
Differentiable Prompt Makes Pre-trained Language Models Better Few-shot LearnersPreprint
Zhang, N., Li, L., Chen, X., Deng, S., Bi, Z., Tan, C., Huang, F., & Chen, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13161 · Cited by 155
Democratizing protein language models with parameter-efficient fine-tuning
Sledzieski, S., Kshirsagar, M., Baek, M., Dodhia, R., Lavista Ferres, J., & Berger, B. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2405840121 · Cited by 86
Illuminating protein space with a programmable generative model
Ingraham, J., Baranov, M., Costello, Z., Frappier, V., Ismail, A., Tie, S., Wang, W., Xue, V., Obermeyer, F., Beam, A., et al. (2022) · doi:10.1101/2022.12.01.518682 · Cited by 83
Prompting PaLM for Translation: Assessing Strategies and Performance
Vilar, D., Freitag, M., Cherry, C., Luo, J., Ratnakar, V., & Foster, G. F. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.859 · Cited by 185
Learning from models beyond fine-tuning
Zheng, H., Shen, L., Tang, A., Luo, Y., Hu, H., Du, B., Wen, Y., & Tao, D. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00961-0 · Cited by 139
Paradigm Shift in Natural Language Processing
Sun, T., Liu, X.-Y., Qiu, X., & Huang, X. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1331-6 · Cited by 103
Learning to compose diversified prompts for image emotion classification
Deng, S., Wu, L., Shi, G., Xing, L., Jian, M., Xiang, Y., & Dong, R. (2024) · Computational Visual Media · doi:10.1007/s41095-023-0389-6 · Cited by 92
Information Retrieval meets Large Language Models: A strategic report from Chinese IR community
Ai, Q., Bai, T., Cao, Z., Chang, Y., Chen, J., Chen, Z., Cheng, Z., Dong, S., Dou, Z., Feng, F., et al. (2023) · AI Open · doi:10.1016/j.aiopen.2023.08.001 · Cited by 115
Large Language Models in Finance (FinLLMs)
Lee, J., Stevens, N., & Han, S. C. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10495-6 · Cited by 203
Pu, X., Jia, H., Zheng, L., Wang, F., & Xu, F. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3532690 · Cited by 102
Zhu, X., Zhu, J., Li, H., Wu, X., Li, H., Wang, X., & Dai, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01630 · Cited by 108
InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning
Liang, Y.-S., & Li, W.-J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02231 · Cited by 124
CricaVPR: Cross-Image Correlation-Aware Representation Learning for Visual Place Recognition
Lu, F. H., Lan, X., Zhang, L., Jiang, D., Wang, Y., & Yuan, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01587 · Cited by 86
ChatGPT as Your Vehicle Co-Pilot: An Initial Attempt
Wang, S., Zhu, Y., Li, Z., Wang, Y., Li, L., & He, Z. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3325300 · Cited by 92
FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model
Wu, F., Li, Z., Li, Y., Ding, B., & Gao, J. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671897 · Cited by 104
Prompt Learning for News Recommendation
Zhang, Z., & Wang, B. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591752 · Cited by 100
Chen, Y., Li, J., Shan, S., Wang, M., & Hong, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3453443 · Cited by 89
Fake news detection via knowledgeable prompt learning
Jiang, G., Liu, S., Zhao, Y., Sun, Y., & Zhang, M. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103029 · Cited by 84
Li, H., Zhu, C., Zhang, Y., Sun, Y., Shui, Z., Kuang, W., Zheng, S., & Yang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00720 · Cited by 86
Pro-Tuning: Unified Prompt Tuning for Vision Tasks
Nie, X., Ni, B., Chang, J., Meng, G., Huo, C., Xiang, S., & Tian, Q. C. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3327605 · Cited by 95
FedPrompt: Communication-Efficient and Privacy-Preserving Prompt Tuning in Federated Learning
Zhao, H., Du, W., Li, F., Li, P., & Liu, G. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095356 · Cited by 114
Kim, B., Kim, H., Lee, S., Lee, G., Kwak, D., Hyeon, J. D., Park, S., Kim, S., Kim, S., Seo, D., et al. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.274 · Cited by 120
Metadata
- Authors
- Lester, B., Al-Rfou, R., & Constant, N.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 3045-3059
- arXiv
- 2104.08691
- OpenAlex
- W3152956381
- Cited by
- 3,092
- References
- 49
- Citations by year
- 2026: 512 · 2025: 889 · 2024: 754 · 2023: 649 · 2022: 246 · 2021: 40
- FWCI
- 195.07
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- prompt tuning, soft prompts, parameter-efficient fine-tuning, frozen language models, few-shot learning, model size, prefix tuning, T5, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
