The DOI matches its DataCite record; 58 of the 58 references it cites were found in a registry or an index.
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Open accessAbstract
Large language models (LMs) are able to in-context learn -- perform a new task via inference alone by conditioning on a few input-label pairs (demonstrations) and making predictions for new inputs. However, there has been little understanding of how the model learns and which aspects of the demonstrations contribute to end task performance. In this paper, we show that ground truth demonstrations are in fact not required -- randomly replacing labels in the demonstrations barely hurts performance on a range of classification and multi-choce tasks, consistently over 12 different models including GPT-3. Instead, we find that other aspects of the demonstrations are the key drivers of end task performance, including the fact that they provide a few examples of (1) the label space, (2) the distribution of the input text, and (3) the overall format of the sequence. Together, our analysis provides a new way of understanding how and why in-context learning works, while opening up new questions about how much can be learned from large language models through inference alone.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Min et al. [44] has shown that rather than learning new tasks, in-context learning [42] works by activating patterns from the training data.”
From RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics (2024) · cites this paper as [44]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Self-citationExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Self-citationChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture notes in computer science · doi:10.1007/11736790_9 · Cited by 1,674
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P. E., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1260 · Cited by 1,602
Automatically Constructing a Corpus of Sentential Paraphrases.
Dolan, W. B., & Brockett, C. (2005) · Cited by 1,101
Research works printed but not linked: 4
- Michael Chen, Mike D'Arcy, Alisa Liu, Jared Fer- nandez, and Doug Downey. 2019. CODAH: An adversarially-authored question answering dataset for common sense. In Proceedings of the 3rd Work- shop on Evaluating Vector Space Representations for NLP.
- Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment chal- lenge. In Machine Learning Challenges Workshop.
- Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, and Grigorios Tsoumakas. 2020. Ethos: an online hate speech detection dataset. ArXiv.
- Wei Wang, Vincent W Zheng, Han Yu, and Chunyan Miao. 2019b. A survey of zero-shot learning: Set- tings, methods, and applications. ACM Transactions on Intelligent Systems and Technology (TIST).
Cited by
Sorted by citations
Items 401–500 of 758
Detecting Misinformation by Uncovering Commonsense Conflicts With LLM Workflows
Wang, B., Li, X., Chang-chun, L., Zhao, B., Guan, R., Wu, L. Y., & Han, J. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3650588 · Cited by 4
Agentic Recommender Systems: A Systematic Literature Review
Portugal, I. D. S., Alencar, P., & Cowan, D. D. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3657900 · Cited by 3
Enhancing Weakly Supervised Multimodal Video Anomaly Detection Through Text Guidance
Sun, S., Hua, J., Feng, J., & Gong, X. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668927 · Cited by 3
DemoShapley: Valuation of Demonstrations for In-Context Learning
Xie, S., Luo, M., Stern, C., Du, M.-N., & Lu, C. (2024) · BigData Congress [Services Society] · doi:10.1109/bigdata66926.2025.11402298 · Cited by 8
Xu, N., Zhang, Q., Ye, J., Zhang, M., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.931 · Cited by 5
Yang, Z., Dai, D., Wang, P., & Sui, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.880 · Cited by 5
BYOC: Personalized Few-Shot Classification with Co-Authored Class Descriptions
Bohra, A., Verkes, G., Harutyunyan, A., Weinberger, P., & Campagna, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.933 · Cited by 8
Mao, T., Yoshie, O., Fu, J., & Mao, W.-X. (2023) · Neural computing & applications (Print) · doi:10.1007/s00521-023-09250-0 · Cited by 4
Wang, Q., Ding, K., Luo, X., & Xu, R. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657932 · Cited by 3
Defect Correction Method for Software Requirements Text Using Large Language Models
Zhang, L., Ouyang, L., & Xu, Z. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651284 · Cited by 3
Few-Shot Optimization for Sensor Data Using Large Language Models: A Case Study on Fatigue Detection
Ronando, E., & Inoue, S. (2025) · Sensors · doi:10.3390/s25113324 · Cited by 3
Santra, P., Ghosh, M., Ganguly, D., Basuchowdhuri, P., & Naskar, S. K. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70021 · Cited by 5
Du, Y., Chen, K., Zhan, Y., Low, C. H., Islam, M., Guo, Z., Jin, Y., Chen, G., & Heng, P. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3581108 · Cited by 4
Design and Evaluation of LLM-Based Conversational Virtual Characters to Assist Adults with ADHD
Li, T., Hu, X., & Xu, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93848-1_24 · Cited by 3
Overcoming classic challenges for artificial neural networks by providing incentives and practice
Irie, K., & Lake, B. M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01121-8 · Cited by 10
Fang, J.-F., Wang, W., Zhang, L., & Lv, F. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755653 · Cited by 3
Gradient Ascent Post-training Enhances Language Model Generalization
Yoon, D., Jang, J., Kim, S., & Seo, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.74 · Cited by 3
Hakimov, S., & Schlangen, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.894 · Cited by 5
OpenBMB: Big Model Systems for Large-Scale Representation Learning
Zeng, G., Han, X., Zhang, Z., Liu, Z., Lin, Y., & Sun, M. (2023) · Springer eBooks · doi:10.1007/978-981-99-1600-9_13 · Cited by 3
Monotonicity Reasoning in the Age of Neural Foundation Models
Chen, Z., & Gao, Q. (2023) · Journal of Logic Language and Information · doi:10.1007/s10849-023-09411-3 · Cited by 3
Tudi, M. R., Na, J., Liu, M., Chen, H., Dai, Y., & Li, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8085-7_5 · Cited by 3
Analyzing Pre-trained and Fine-tuned Language Models
Mosbach, M. (2023) · BIGPICTURE · doi:10.18653/v1/2023.bigpicture-1.10 · Cited by 7
Tian, Q., Zhu, H., Wang, L., Li, Y., & Lan, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.114 · Cited by 4
Robustness of Named-Entity Replacements for In-Context Learning
Goodarzi, S., Kagita, N., Minn, D., Wang, S., Dessì, R., Toshniwal, S., Williams, A., Lanchantin, J., & Sinha, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.728 · Cited by 5
Demonstration-Based and Attention-Enhanced Grid-Tagging Network for Mention Recognition
Jia, H., Huang, J., Kang, Z., Mao, Y., Zhou, H., Ren, L., Jia, Y., & Xu, W. (2024) · Electronics · doi:10.3390/electronics13020261 · Cited by 4
In-Context Prompt Editing for Conditional Audio Generation
Chang, E., Lin, P.-J., Li, Y., Srinivasan, S., Lan, G. L., Kant, D., Shi, Y., Iandola, F., & Chandra, V. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446431 · Cited by 4
A few-shot learning method based on knowledge graph in large language models
Wang, F., Shi, D., Aguilar, J., & Cui, X. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00699-3 · Cited by 3
Park, G., & Choi, S. (2025) · Journal of Curriculum and Evaluation · doi:10.29221/jce.2025.28.1.215 · Cited by 5
Triplet-based contrastive method enhances the reasoning ability of large language models
Chen, H., Zhu, J., Wang, W., Zhu, Y., & Xi, L. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07056-6 · Cited by 3
Liu, J., Su, Y., & Seth, P. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719821 · Cited by 3
Improving optimal prompt learning through multilayer fusion and latent dirichlet allocation
Chen, Q., Korneder, J., Rawashdeh, O. A., Wang, Y., & Louie, W.-Y. G. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1579990 · Cited by 3
A Survey of Visual Analytics Research for Improving Training Data Quality
Yang, W., Chen, C., Zhu, J., Li, L., Liu, P., & Liu, S. (2023) · Journal of Computer-Aided Design & Computer Graphics · doi:10.3724/sp.j.1089.2023.2023-00321 · Cited by 4
Enhancing Autonomous Driving Safety with Collision Scenario Integration
Wang, Z., Lan, S., Sun, X., Chang, N., Li, Z., Yu, Z., & Álvarez, J. M. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246713 · Cited by 10
RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
Yuan, W., Duan, J., Blukis, V., Pumacay, W., Krishna, R., Murali, A., Mousavian, A., & Fox, D. (2024) · arXiv · doi:10.48550/arxiv.2406.10721 · Cited by 7
Graph-based event schema induction in open-domain corpus
Yan, K., Liu, W., Xie, S., & Peng, Y. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2155 · Cited by 2
Chain-of-Program Prompting with Open-Source Large Language Models for Text-to-SQL
Xu, B., Li, S., Wu, Y., Wei, S., Du, M., Wang, H., & Song, H. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650552 · Cited by 3
Harnessing GPT for Data Transformation Tasks
Ghazzai, S., Grigori, D., Benatallah, B., & Rébaï, R. (2024) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws62655.2024.00160 · Cited by 2
MIRA : A Method of Federated Multi-Task Learning for Large Language Models
Elbakary, A., Issaid, C. B., ElBatt, T., Seddik, K. G., & Bennis, M. (2025) · IEEE Networking Letters · doi:10.1109/lnet.2025.3539810 · Cited by 2
Object Positions Interpretation System for Service Robots Through Targeted Object Marking
Yamao, K., Kanaoka, D., Isomoto, K., & Tamukoh, H. (2025) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii59315.2025.10870994 · Cited by 2
Fan, Z., Chen, C., & Luo, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127434 · Cited by 2
Leveraging Concise Concepts With Probabilistic Modeling for Interpretable Visual Recognition
Zhang, Y., Liu, C., Liu, Y., Gao, Y., Lu, Z., Xie, H., & Zhang, Y. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557677 · Cited by 2
Mobile Traffic Prediction Using LLMs With Efficient In-Context Demonstration Selection
Zhang, H., Sediq, A. B., Afana, A., & Erol‐Kantarci, M. (2025) · IEEE Transactions on Communications · doi:10.1109/tcomm.2025.3579703 · Cited by 2
Szczęsny, A., Markiewicz, M., Radliński, Ł., & Kazienko, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97635-3_7 · Cited by 2
Li, Y., & Fowlie, M. (2025) · LDV-Forum/Journal for language technology and computational linguistics · doi:10.21248/jlcl.38.2025.285 · Cited by 2
Zero-Shot Knowledge-Based Visual Question Answering with Frozen Language Models
Liu, J., Zhang, L., Cao, C., Shi, Y., Mu, C., & Li, J. (2025) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020032 · Cited by 2
Active Knowledge Structuring for Large Language Models in Materials Science Text Mining
Zhang, X. Q., Yuan, J., Zhang, P., Liu, J., & Li, L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.36 · Cited by 2
HMF: Enhancing reentrancy vulnerability detection and repair with a hybrid model framework
Li, M., Shen, Q., Ren, X., Fu, H., Li, Z., & Sun, J. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00546-0 · Cited by 2
Augmenting, Not Replacing: The Role of LLMs in Human-Centric Formal RE
Halili, S., Spoletini, P., & Grubb, A. M. (2025) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re63999.2025.00029 · Cited by 2
Avis, G. R., Nocco, S., Zhao, A. Q., & Cremaschi, M. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2581258 · Cited by 2
Beyond Meme Templates: Limitations of Visual Similarity Measures in Meme Matching
Hazman, M., McKeever, S., & Griffith, J. (2025) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta66025.2025.11222070 · Cited by 2
Barlow, S. H., Chicklore, S., He, Y., Sebastien Ourselin, A., Wagner, T., Barnes, A., & Cook, G. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1702082 · Cited by 3
Yang, X., Hu, B., & Zhao, J. (2025) · Land · doi:10.3390/land14122371 · Cited by 2
Cherupalli, R., Grubbs, H., Chua, Y. T., Pei, W., Moore, T., & Warner, G. (2025) · APWG Symposium on Electronic Crime Research (eCrime) · doi:10.1109/ecrime66972.2025.11327876 · Cited by 2
Information Retrieval and In-Context Learning for Low-Resource Named Entity Recognition
Shiraishi, E., Camargo, R. Y. D., Silva, H. L. P., & Prati, R. C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15984-7_10 · Cited by 2
Cosentino, C., Folino, F., Marozzo, F., & Pontieri, L. (2026) · Array · doi:10.1016/j.array.2026.100705 · Cited by 4
Pang, R. Y., Feng, K. J. K., Feng, S., Li, C., Shi, W., Tsvetkov, Y., Heer, J., & Reinecke, K. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789091 · Cited by 31
Leveraging large language models for automated knowledge extraction from geological reports
Ge, Q., Li, P., Li, J., Sun, H., & Liu, Z. (2026) · Journal of Rock Mechanics and Geotechnical Engineering · doi:10.1016/j.jrmge.2025.12.038 · Cited by 2
Collective Human Opinions in Semantic Textual Similarity
Wang, Y., Tao, S., Xie, N., Yang, H., Baldwin, T., & Verspoor, K. M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00584 · Cited by 7
Holistic Spatial Reasoning for Chinese Spatial Language Understanding
Zhao, Y., & Wei, J. (2023) · Applied Sciences · doi:10.3390/app132111712 · Cited by 2
Prompterator: Iterate Efficiently towards More Effective Prompts
Sučík, S., Skala, D., Švec, A., Hraška, P., & Šuppa, M. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.43 · Cited by 3
Dissecting In-Context Learning of Translations in GPT-3
Raunak, V., Menezes, A., & Awadalla, H. H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.61 · Cited by 8
DemoNSF: A Multi-task Demonstration-based Generative Framework for Noisy Slot Filling Task
Dong, G., Hui, T., Gongque, Z., Zhao, J., Guo, D., Zhao, G., He, K., & Xu, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.705 · Cited by 4
He, M., & Bai, Y. (2023) · International Conference on Communication Networks and Machine Learning (CNML) · doi:10.1145/3640912.3640993 · Cited by 2
Survey in characterizing semantic change
de Sá, J. M. C., Silveira, M. D., & Pruski, C. (2026) · Natural language processing. · doi:10.1017/nlp.2026.10023 · Cited by 10
HICL: Hashtag-Driven In-Context Learning for Social Media Natural Language Understanding
Tan, H., Xu, C., Li, J., Zhang, Y., Fang, Z., Chen, Z., & Lai, B. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3384987 · Cited by 5
Kong, D., Ma, L., Chen, W., Li, Y., You, S., Zhu, S., Wang, Q., & Li, X. (2023) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc59590.2023.10507486 · Cited by 4
Combating Missed Recalls in E-commerce Search: A CoT-Prompting Testing Approach
Wu, S., Hu, Y., Wang, Y., Gu, J., Meng, J., Fan, L., Luan, Z., Wang, X., & Zhou, Y. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663842 · Cited by 3
ExPrompt: Augmenting Prompts Using Examples as Modern Baseline for Stance Classification
Qudus, U., Röder, M., Vollmers, D., & Ngomo, A.-C. N. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679923 · Cited by 2
Chaudhary, S. (2024) · Conference on AI, Science, Engineering, and Technology (AIxSET) · doi:10.1109/aixset62544.2024.00014 · Cited by 2
VAGen: waterbody segmentation with prompting for visual in-context learning
Zhao, J., Yabuki, N., & Fukuda, T. (2024) · AI in Civil Engineering · doi:10.1007/s43503-024-00042-6 · Cited by 2
Ghanem, H., & Cruz, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-81221-7_3 · Cited by 2
Zhuang, N., Wei, X., Li, J., Wang, X., Wang, C., Wang, L., & Dang, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888364 · Cited by 2
SGEU: enhancing LLM reasoning via backward exemplar generation and verification
Wang, Z., Zhou, X., Yang, Y., Ma, B., Wang, L., & Dong, R. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06529-8 · Cited by 1
Programming Large Language Models
Calamo, M., & Rossi, J. (2025) · Springer eBooks · doi:10.1007/978-3-031-92285-5_6 · Cited by 2
Jagpal, D., Chen, X., & Namboodiri, V. P. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01698 · Cited by 2
An Empirical Study of Production Incidents in Generative AI Cloud Services
Yan, H., Chen, Y., Ma, M., Wen, M., Lu, S., Zhang, S., Xu, T., Wang, R., Bansal, C., Rajmohan, S., et al. (2025) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre66568.2025.00044 · Cited by 2
Hijacking Large Language Models via Adversarial In-Context Learning
Zhou, X., Yao, Q., Zade, S. Z., Khanduri, P., & Zhu, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37673-2_13 · Cited by 49
Prompt and Instruction-Based Tuning for Response Generation in Conversational Question Answering
Xing, Y., & Liu, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35320-8_11 · Cited by 2
Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation
Wang, S., Wei, K., Zhang, H., Li, Y., & Wu, W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.93 · Cited by 3
Transferring Procedural Knowledge Across Commonsense Tasks
Jiang, Y., Ilievski, F., & Ma, K. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230391 · Cited by 4
Assessing and Enhancing LLMs: A Physics and History Dataset and One-More-Check Pipeline Method
He, C., Li, C., Han, T., & Shen, L. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8178-6_38 · Cited by 2
Lu, Y., Yu, H. Q., Wang, Z., & Zhou, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.374 · Cited by 2
Task-Level Thinking Steps Help Large Language Models for Challenging Classification Task
Du, C., Tian, J., Liao, H., Chen, J., He, H., & Jin, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.150 · Cited by 2
Social Networks and Large Language Models for Division I Basketball Game Winner Prediction
Sprint, G. (2024) · IEEE Access · doi:10.1109/access.2024.3403490 · Cited by 2
Evolutionary Prompt Design for LLM-Based Post-ASR Error Correction
Sachdev, R., Wang, Z.-Q., & Yang, C.-H. H. (2025) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips66314.2025.11261248 · Cited by 2
Distilling Mathematical Reasoning Capabilities into Small Language ModelsPreprint
Zhu, X., Li, J., Liu, Y., Ma, C., & Wang, W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4782551 · Cited by 2
On the Underspecification of Situations in Open-domain Conversational Datasets
Otani, N., Araki, J., Kim, H., & Hovy, E. H. (2023) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2023.nlp4convai-1.2 · Cited by 2
Similarity-Based Prompt Construction for Large Language Model in Medical Tasks
Liu, G., Pan, M., Ma, Z., Gu, M., Yang, L., & Qin, J. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-1717-0_6 · Cited by 1
Wan, L. J., Ye, H., Wang, J., Jha, M., & Chen, D. (2024) · IEEE LLM Aided Design Workshop (LAD) · doi:10.1109/lad62341.2024.10691860 · Cited by 1
PatUntrack: Automated Generating Patch Examples for Issue Reports without Tracked Insecure Code
Jiang, Z., Shi, L., Yang, G., & Wang, Q. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3694982 · Cited by 1
Can Language Models Pretend Solvers? Logic Code Simulation with LLMs
Chen, M., Li, G., Wu, L.-I., Liu, R., Su, Y., Chang, X., & Xue, J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0602-3_6 · Cited by 6
Can a Large Language Model Learn Matrix Functions In Context?
Goulart, P., & Papalexakis, E. E. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825481 · Cited by 1
Towards the Effect of Examples on In‐Context Learning: A Theoretical Case Study
He, P., Cui, Y., Xu, H., Liu, H., Yamada, M., Tang, J., & Xing, Y. (2025) · Stat · doi:10.1002/sta4.70045 · Cited by 3
Reliable Academic Conference Question Answering: A Study Based on Large Language Model
Huang, Z., Li, J., Jin, L. H., Wang, J., Tu, M., Yin, H., Liu, Z., Meng, J., & Zhang, W. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-1809-5_14 · Cited by 1
Insufficient task description can impair in-context learning: A study from information perspective
Xuanyuan, M., Yang, T., Fu, J., Zhao, S., & Wang, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103116 · Cited by 1
Ji, D., Luo, H., Bao, Z., & Culpepper, J. S. (2025) · The VLDB Journal · doi:10.1007/s00778-025-00917-9 · Cited by 4
Wu, P. Y. (2025) · ACM Web Science Conference (WebSci) · doi:10.1145/3717867.3717913 · Cited by 1
Diverse Prompts: Illuminating the Prompt Space of Large Language Models with MAP-Elites
Santos, G. M., Julia, R. M. D. S., & do Nascimento, M. Z. (2025) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec65147.2025.11043024 · Cited by 2
Metadata
- Authors
- Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L.
- Affiliations
- University of Washington (US)
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Pages
- 11048-11064
- arXiv
- 2202.12837
- OpenAlex
- W4385567149
- Cited by
- 752
- References
- 57
- Citations by year
- 2026: 135 · 2025: 227 · 2024: 179 · 2023: 191 · 2022: 20
- FWCI
- 85.30
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2044660)
- Office of Naval Research (N00014-18, N00014-18-1-2826, ONR-N00014, N00014)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- in-context learning, demonstrations, large language models, label noise, input distribution, label space, GPT-3, classification tasks, multiple-choice tasks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review