The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Demystifying Prompts in Language Models via Perplexity Estimation
Open accessAbstract
Language models can be prompted to perform a wide variety of tasks with zero-and few-shot incontext learning.However, performance varies significantly with the choice of prompt, and we do not yet understand why this happens.In this paper, we analyze the factors that contribute to this variance and establish a new empirical hypothesis: the performance of a prompt is predicted by the extent to which the model is familiar with the language it contains.Over a wide range of tasks, we show that the lower the perplexity of the prompt, the better it is able to perform the task, when considering reasonable prompts that are related to it.As part of our analysis, we also devise a method to automatically extend a small seed set of manually written prompts by paraphrasing with GPT3 and backtranslation.This larger set allows us to verify that perplexity is a strong predictor of the success of a prompt and we show that the lowest perplexity prompts are consistently effective.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This provides a clear signal for detection methods like Perplexity (Gonen et al., 2023) and Min-K% Prob (Shi et al., 2024), which are built upon this likelihood principle.”
From Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models (2025) · cites this paper as Gonen et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 4,481
DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia
Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P. N., Hellmann, S., Morsey, M., van Kleef, P., Auer, S., et al. (2015) · Semantic Web · doi:10.3233/sw-140134 · Cited by 3,737
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 3,710
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473
No Language Left Behind: Scaling Human-Centered Machine TranslationPreprint
Costa-jussà, M., Cross, J., Çelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., et al. (2022) · arXiv.org · doi:10.48550/arxiv.2207.04672 · Cited by 2,176
5.7 The WORDNET lexical database of English
Ooi, V. B. Y. (2019) · Edinburgh University Press eBooks · doi:10.1515/9781474471459-050 · Cited by 2,035
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380
Research works printed but not linked: 2
- George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41.
- John Wieting, Jonathan Mallinson, and Kevin Gimpel. 2017. Learning paraphrastic sentence embeddings from back-translated bitext. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 274-285.
Cited by
Sorted by citations
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.64 · Cited by 649
Security and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
Gao, Y., Li, R., Croxford, E., Caskey, J., Patterson, B. W., Churpek, M. M., Miller, T. A., Dligach, D., & Afshar, M. (2024) · JMIR AI · doi:10.2196/58670 · Cited by 169
State of What Art? A Call for Multi-Prompt LLM Evaluation
Mizrahi, M., Kaplan, G., Malkin, D., Dror, R., Shahaf, D., & Stanovsky, G. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00681 · Cited by 113
Prompting meaning: a hermeneutic approach to optimising prompt engineering with ChatGPT
Henrickson, L., & Meroño-Peñuela, A. (2023) · AI & Society · doi:10.1007/s00146-023-01752-8 · Cited by 136
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4
Chang, K. K., Cramer, M. H., Soni, S., & Bamman, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.453 · Cited by 176
Jaffal, N. O., Alkhanafseh, M., & Mohaisen, D. (2025) · AI · doi:10.3390/ai6090216 · Cited by 69
The Art of Creative Inquiry—From Question Asking to Prompt Engineering
Sasson-Lazovsky, G., Raz, T., & Kenett, Y. N. (2024) · The Journal of Creative Behavior · doi:10.1002/jocb.671 · Cited by 73
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2024) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 133
Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies
Nan, L., Zhao, Y., Zou, W., Ri, N., Tae, J., Zhang, E., Cohan, A., & Radev, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.996 · Cited by 102
Yang, L., Wang, Z., Li, Z., Na, J., & Yu, J. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103724 · Cited by 46
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
Zhou, K., Jurafsky, D., & Hashimoto, T. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.335 · Cited by 38
Afshar, M., Resnik, F., Ryan, M. M., Hintzke, J., Lemmon, K. K., Sullivan, A. G., Shah, T., Stordalen, A., Oberst, M., Dambach, J., et al. (2025) · NEJM AI · doi:10.1056/aidbp2401267 · Cited by 40
Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains
Cheng, Y., Chen, J., Huang, Q. Y., Xing, Z., Xu, X., & Lu, Q. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3638247 · Cited by 35
Language models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K. M., & Cohn, T. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.10 · Cited by 69
Language models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K., & Cohn, T. (2023) · arXiv · doi:10.48550/arxiv.2306.08189 · Cited by 34
Active Learning Principles for In-Context Learning with Large Language Models
Margatina, K., Schick, T., Αλέτρας, Ν., & Dwivedi-Yu, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.334 · Cited by 72
The language of prompting: What linguistic properties make a prompt successful?
Leidinger, A., van Rooij, R., & Shutova, E. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.618 · Cited by 68
MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
Zhang, C., Lin, K., Yang, Z., Wang, J., Li, L., Lin, C.-C., Liu, Z., & Wang, L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01295 · Cited by 24
Evaluation of African American Language Bias in Natural Language Generation
Deas, N., Grieser, J. A., Kleiner, S., Patton, D. U., Turcan, E., & McKeown, K. R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.421 · Cited by 41
Dr.ICL: Demonstration-Retrieved In-context Learning
Luo, M., Xu, X., Dai, Z., Pasupat, P., Kazemi, M., Baral, C. R., Imbrasaitė, V., & Zhao, V. Y. (2024) · Data Intelligence · doi:10.3724/2096-7004.di.2024.0012 · Cited by 86
Aligning generalization between humans and machines
Ilievski, F., Hammer, B., van Harmelen, F., Paaßen, B., Saralajew, S., Schmid, U., Biehl, M. L., Bolognesi, M., Dong, X., Gashteovski, K., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01109-4 · Cited by 37
Large Language Models and Medical Knowledge Grounding for Diagnosis PredictionPreprint
Gao, Y., Li, R., Croxford, E., Tesch, S., To, D., Caskey, J., Patterson, B. W., Churpek, M. M., Miller, T. A., Dligach, D., et al. (2023) · medRxiv · doi:10.1101/2023.11.24.23298641 · Cited by 24
Calle, P., Shao, R., Liu, Y., Hébert, E. T., Kendzor, D. E., Neil, J. M., Businelle, M. S., & Pan, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641965 · Cited by 20
Instruct Me More! Random Prompting for Visual In-Context Learning
Zhang, J., Wang, B., Li, L., Nakashima, Y., & Nagahara, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00258 · Cited by 20
Morris, M. R. (2024) · Communications of the ACM · doi:10.1145/3673861 · Cited by 17
PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Zou, W., Geng, R., Wang, B., & Jia, J. (2024) · arXiv · doi:10.48550/arxiv.2402.07867 · Cited by 16
Data Curation Alone Can Stabilize In-context Learning
Chang, T.-Y., & Jia, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.452 · Cited by 37
CoLE: A collaborative legal expert prompting framework for large language models in law
Li, B., Fan, S., Zhu, S., & Wen, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113052 · Cited by 15
Hidayatullah, A. F., Apong, R. A. A. H. M., Lai, D. T. C., & Qazi, A. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01444-9 · Cited by 23
GENERATIVE PRE-TRAINED TRANSFORMER 3
Golubenko, O., & Pidmohylʹnyy, O. O. (2022) · ITSynergy · doi:10.53920/its-2022-2-2 · Cited by 15
Toward Human Readable Prompt Tuning: Kubrick’s The Shining is a good movie, and a good prompt too?
Shi, W., Han, X., Gonen, H., Holtzman, A., Tsvetkov, Y., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.733 · Cited by 44
Self-citationimpact of sample selection on in-context learning for entity extraction from scientific writing
Bölücü, N., Rybinski, M., & Wan, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.338 · Cited by 16
Can deepfakes be used to study emotion perception? A comparison of dynamic face stimuli
Becker, C., Conduit, R., Chouinard, P. A., & Laycock, R. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02443-y · Cited by 17
Hou, R., Wang, X., Zhang, W., Song, Z., Wang, K., Chen, Y., Liu, J., & Ruan, T. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3529682 · Cited by 15
Pecher, B., Srba, I., & Bieliková, M. (2024) · ACM Computing Surveys · doi:10.1145/3691339 · Cited by 19
Navigating the Risks of Using Large Language Models for Text Annotation in Social Science Research
Lin, H., & Zhang, Y. (2025) · Social Science Computer Review · doi:10.1177/08944393251366243 · Cited by 22
“AI, will you help?” How learners use Artificial Intelligence when writing
Krajka, J., & Olszak, I. (2024) · XLinguae · doi:10.18355/xl.2024.17.01.03 · Cited by 11
Xu, Z., & Wall, E. (2024) · Visualization · doi:10.1109/vis55277.2024.00033 · Cited by 24
Performance of Large Language Models in a Computer Science Degree Program
Krüger, T., & Gref, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-50485-3_40 · Cited by 11
Adaptive Prompt Routing for Arbitrary Text Style Transfer with Pre-trained Language Models
Liu, Q., Qin, J., Ye, W., Mou, H. L., He, Y., & Wang, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29832 · Cited by 11
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations
Chen, W., Wu, C.-K., Chen, Y.-N., & Chen, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.968 · Cited by 33
Croxford, E., Gao, Y., Patterson, B. W., To, D., Tesch, S., Dligach, D., Mayampurath, A. M., Churpek, M. M., & Afshar, M. (2024) · medRxiv · doi:10.1101/2024.03.20.24304620 · Cited by 13
Security and Privacy of Large Language Models: Threat Taxonomy, Ethical Implications, and Governance
Pribisalić, M., & Martinčić-Ipšić, S. (2026) · AI · doi:10.3390/ai7050152 · Cited by 9
Metric-Based In-context Learning: A Case Study in Text Simplification
Vadlamannati, S., & Şahin, G. G. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.18 · Cited by 8
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting
Ye, X., & Durrett, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.41 · Cited by 11
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis
Yang, S., Kim, J., Jang, J., Ye, S., Lee, H., & Seo, M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00666 · Cited by 12
Data Similarity is Not Enough to Explain Language Model Performance
Yauney, G., Reif, E., & Mimno, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.695 · Cited by 13
Security and privacy in LLMs: A comprehensive survey of threats and mitigation strategies
Berini, A. D. E., Jamil, N., Benrazek, A., Lakas, A., Ismail, L., Ferrag, M. A., & Lam, K. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104241 · Cited by 14
Zhao, Y., Zhang, Y., Sui, D., Cao, L., Zhao, J., & Liu, K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.331 · Cited by 13
LLM-driven Instruction Following: Progresses and Concerns
Yin, W., Ye, Q., Liu, P., Ren, X., & Schütze, H. (2023) · doi:10.18653/v1/2023.emnlp-tutorial.4 · Cited by 8
Bridging Information-Theoretic and Geometric Compression in Language Models
Cheng, E., Kervadec, C., & Baroni, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.762 · Cited by 17
GenKP: generative knowledge prompts for enhancing large language models
Li, X., Peng, S., Yada, S., Wakamiya, S., & Aramaki, E. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06318-3 · Cited by 6
Context Example Selection for LLM Generated Relevance Assessments
McKechnie, J., McDonald, G., & Macdonald, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_19 · Cited by 6
Hint-Enhanced In-Context Learning Wakes Large Language Models Up For Knowledge-Intensive Tasks
Wang, Y., Guo, Q., Ni, X., Shi, C., Liu, L., Jiang, H., & Yang, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447527 · Cited by 9
Large Language Models for Recommender Systems: A Problem-Driven Survey
Guan, Z., Zhong, W., Liu, W., Liu, Y., Kou, H., Chi, X., & Zhao, X. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2025.9010137 · Cited by 5
Li, B., Liu, T., Ma, J., Dai, J., & Liao, P. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111074 · Cited by 4
In-Context Demonstration Selection with Cross Entropy Difference
Iter, D., Pryzant, R., Xu, R., Wang, S., Liu, Y., Yi‐chong, X., & Zhu, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.81 · Cited by 13
CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation
Aswanth, M., Puduppully, R., Dabre, R., & Kunchukuttan, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.519 · Cited by 9
Mo, Y., Tang, M., Lin, R., Zhou, B., & Li, X. (2025) · IEEE Access · doi:10.1109/access.2025.3582519 · Cited by 5
Language Model Guided Reinforcement Learning in Quantitative Trading
Darmanin, A., & Vella, V. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11390898 · Cited by 3
Yang, Z., Dai, D., Wang, P., & Sui, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.880 · Cited by 5
EEG correlates of static and dynamic face perception: The role of naturalistic motion
Becker, C., Conduit, R., Chouinard, P. A., & Laycock, R. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.108986 · Cited by 3
What Is Wrong with My Model? Identifying Systematic Problems with Semantic Data Slicing
Yang, C., Hong, Y., Lewis, G. A., Wu, T., & Kästner, C. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695033 · Cited by 5
E-InMeMo: Enhanced Prompting for Visual In-Context Learning
Zhang, J., Wang, B., Liu, H., Li, L., Nakashima, Y., & Nagahara, H. (2025) · Journal of Imaging · doi:10.3390/jimaging11070232 · Cited by 4
Engagement-Driven Content Generation with Large Language Models
Coppolillo, E., Cinus, F., Minici, M., Bonchi, F., & Manco, G. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736932 · Cited by 11
Frugal Prompting for Dialog Models
Santra, B., Basak, S., De, A., Gupta, M., & Goyal, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.290 · Cited by 3
Robustness of Named-Entity Replacements for In-Context Learning
Goodarzi, S., Kagita, N., Minn, D., Wang, S., Dessì, R., Toshniwal, S., Williams, A., Lanchantin, J., & Sinha, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.728 · Cited by 5
Input-oriented demonstration learning for hybrid evidence fact verification
Chen, C., Chen, W., Zheng, J., Luo, A., Cai, F., & Zhang, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123191 · Cited by 5
Afshar, M., Gao, Y., Wills, G. J., Wang, J., Churpek, M. M., Westenberger, C. J., Kunstman, D., Gordon, J. E., Liao, F., & Patterson, B. W. (2024) · medRxiv · doi:10.1101/2024.01.23.24301692 · Cited by 3
Systematic survey of various prompt optimization methods and their classifications
Jain, A. M., & Jindal, M. (2025) · International Conference on Computing and Artificial Intelligence (ICCAI) · doi:10.1109/iccai66501.2025.00085 · Cited by 3
Mobadersani, M., Candaş, A. B., Kuruoğlu, M., & Tokdemir, O. B. (2025) · Buildings · doi:10.3390/buildings16010120 · Cited by 3
Gupta, P., Jain, N., & Bhat, A. (2024) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic60222.2024.10575336 · Cited by 2
LLM Based Physical Verification Runset Generator
Francisco, L., & Arikati, S. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1145/3670474.3685976 · Cited by 2
Tiwari, A., Kadian, K., Garg, N., Kaur, S., & Dwivedi, V. K. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-75164-6_4 · Cited by 2
ContextualCoder: Adaptive In-Context Prompting for Programmatic Visual Question Answering
Shen, R., Inoue, N., Guan, D., Cai, R., Kot, A. C., & Shinoda, K. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3543043 · Cited by 1
Pragmatic inference and mapping for conversational implicature
Hu, J., Liu, Y., Ma, L., Zhang, W.-N., & Liu, T. (2025) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02741-7 · Cited by 2
Zhang, J., Wada, S., Sugimoto, K., Niitsu, T., Fukushima, K., Kida, H., Wang, B., Konishi, S., Okada, K., Nakashima, Y., et al. (2025) · Journal of Imaging · doi:10.3390/jimaging11100353 · Cited by 1
Hu, J., Bi, W., Zhou, Y., Zheng, J., Zhang, S., Chu, H., Wang, L., & Li, Q. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130118 · Cited by 1
TaskEval: Assessing Difficulty of Code Generation Tasks for Large Language Models
Tambon, F., Nikanjam, A., Zid, C., Khomh, F., & Antoniol, G. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3773285 · Cited by 2
TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
He, X., Yu, H., Sun, Q., Cheng, A., Zhang, T., Liu, C., & Guo, S. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209076 · Cited by 1
PRSM: A Measure to Evaluate CLIP’s Robustness Against Paraphrases
Schlegel, U., Weeber, F., Lan, J. W., & Seidl, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-6963-2_6 · Cited by 1
Aiyappa, R., Senthilmani, S., An, J., Kwak, H., & Ahn, Y. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3540 · Cited by 14
Event-Centric Opinion Mining via In-Context Learning with ChatGPT
Wu, L., Chen, Y., Yang, J., Shi, G., Qi, X., & Deng, S. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-7224-1_7 · Cited by 1
Unnatural language processing: How do language models handle machine-generated prompts?
Kervadec, C., Franzon, F., & Baroni, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.959 · Cited by 1
ReadPrompt: A Readable Prompting Method for Reliable Knowledge Probing
Wang, Z., Ye, L., Wang, H., Kwan, W.-C., Ho, D., & Wong, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.501 · Cited by 1
Open Llama2 Models for the Lithuanian Language
Nakvosas, A., Daniušis, P., & Mulevičius, V. (2025) · Informatica · doi:10.15388/25-infor592 · Cited by 1
LLM Based Physical Verification Runset Generator
Francisco, L., & Arikati, S. (2024) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad62225.2024.10740218 · Cited by 1
Du, Y., Zhang, Q., Fu, S., Hou, Y., & Han, H. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07214-w · Cited by 1
Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
Tao, Y., Wang, T., Dong, Y., Liu, H., Zhang, K., Hu, X., & Li, G. (2025) · arXiv · doi:10.48550/arxiv.2510.09259 · Cited by 1
Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers
Wei, K., Ghosh, S., Menon, R. R., & Srivastava, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.471
Improving Input-label Mapping with Demonstration Replay for In-context Learning
Gong, Z., Liu, J., Wang, Q., Wang, J., Cai, X., Zhao, D., & Yan, R. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.995 · Cited by 2
SMOOTHIE: Label Free Language Model RoutingPreprint
Guha, N., Chen, M. F., Chow, T., Khare, I. S., & Ré, C. (2025) · Qeios · doi:10.32388/f319w4
Probability-Selected Demonstrations for Enhanced Zero-Shot in-Context Learning
Bai, B. (2024) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice63571.2024.10864087
Multi-Modal Relation Extraction Based on Shortest Dependency Path and Large Language ModelsPreprint
Xue, H., Dai, Y., Ma, J., & Guo, X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5131704
Leveraging Correctness-Based Learning for Efficient Reasoning via Automated Process Supervision
Yao, Y., Liu, M., Deng, G., Liu, Z., Wu, D. O., Guo, Z., Wu, H., & Song, L. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3581101
Is a Similar Task Useful for Few-Shot Selection? Aspect Term Extraction Using LLM
Imazato, K., & Shimada, K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97144-0_5
Influencia del formato y la complejidad de prompts en calidad de respuestas de modelos generativos
Vera, W. F. Z., & Carrillo, J. J. M. (2025) · Revista Científica de Informática ENCRIPTAR · doi:10.56124/encriptar.v8i16.011
Qiu, X., Cheng, L., Hao, T., & Tan, X. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3630623
Metadata
- Authors
- Gonen, H., Iyer, S., Blevins, T., Smith, N., & Zettlemoyer, L.
- Affiliations
- University of Washington (US)
- Allen Institute for Artificial Intelligence (US)
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Published
- 2023
- Pages
- 10136-10148
- OpenAlex
- W4389520255
- Cited by
- 109
- References
- 32
- Citations by year
- 2026: 14 · 2025: 33 · 2024: 30 · 2023: 31
- FWCI
- 19.14
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- prompt engineering, language models, in-context learning, zero-shot learning, few-shot learning, paraphrase, back-translation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review