The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index; none of the 3 sources its claims cite is checked for support yet.
Learning To Retrieve Prompts for In-Context Learning
Open accessAbstract
In-context learning is a recent paradigm in natural language understanding, where a large pretrained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters.However, performance has been shown to strongly depend on the selected training examples (termed prompts).In this work, we propose an efficient method for retrieving prompts for in-context learning using annotated data and an LM.Given an inputoutput pair, we estimate the probability of the output given the input and a candidate training example as the prompt, and label training examples as positive or negative based on this probability.We then train an efficient dense retriever from this data, which is used to retrieve training examples as prompts at test time.We evaluate our approach on three sequence-tosequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper proposes Efficient Prompt Retrieval (EPR), which uses a language model to label which training examples are good in-context prompts and trains a dense retriever from that signal. EPR substantially outperforms prior prompt-retrieval baselines on BREAK, MTOP, and SMCALFLOW.
Key findings
- With GPT-NEO as both scoring and inference LM, EPR improves over the best baseline from 26.0 to 31.9 on BREAK, from 57.0 to 64.2 on MTOP, and from 51.4 to 54.3 on SMCALFLOW.
- In the one-shot BREAK setting, EPR reaches 23.0 LF-EM, outperforming the best baseline by 8.5 points and BM25-ORACLE by 5 points.
- When GPT-NEO is used as a proxy scoring LM for larger inference LMs, EPR outperforms BM25 and CBR by 2–8 points across all datasets and pretrained models.
- Prompt-copying analysis shows high abstract copying in MTOP and SMCALFLOW (>80%); on MTOP, 84.5% of examples were abstractly copied, and EPR achieved 71.6% EM on that copied subset versus 64.2% overall.
Methods
- EPR generates training data by retrieving candidate prompts with an unsupervised retriever (BM25 or SBERT; BM25 chosen), scoring each candidate with a scoring LM as Prob(y | candidate, x), and labeling top-k as positive and bottom-k as negative (k=5, L=50).
- It trains a dense retriever with DPR-style contrastive learning using BERT-base input and prompt encoders, then uses FAISS maximum inner-product search at inference to retrieve prompts within the inference LM’s context limit.
- Evaluation covers three structured sequence-to-sequence tasks—BREAK, MTOP, and SMCALFLOW—using GPT-NEO as scoring LM and GPT-NEO, GPT-J, GPT-3, or CODEX as inference LMs, with LF-EM/NEM/EM metrics and a 2,048-token context size.
Limitations
- Paper-stated: the authors note that large language models exhibit bias, and since EPR is trained on signal from such LMs, it might also exhibit these biases.
- Paper-stated: the method scores each training example independently at training time, while at test time the LM observes a sequence of examples; modeling dependence between examples is left to future work.
- Evident from the text: evaluation is limited to three English structured tasks mapping utterances to meaning representations, and the approach relies on a scoring LM as a proxy for the inference LM.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
19 Own result · 11 Own interpretation · 6 No basis given · 3 Not checked yet
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows test results on BREAK and MTOP corroborating that EPR substantially improves performance compared to BM25 and CBR.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“BM25 outperforms SBERT for prompt retrieval, and consequently we use BM25 in all of our supervised approaches to retrieve the set of candidates, Ē.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“EPR substantially outperforms all other baselines.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The candidate set Ē includes good prompt candidates and hard negatives.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“CODEX outperforms GPT-3, which can be explained by the fact that it was trained on code, and our datasets involve mapping to programs or meaning representations.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, accuracy on examples where copying occurred is much higher compared to accuracy where no copying happened.”
§ Learning To Retrieve Prompts for In-Context Learning > Prompt copying
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We test the effect of k, the number of prompts labeled as positive or negative, and L, the number of prompts retrieved by the unsupervised retriever.Table 8 shows that performance is is generally robust w.r.t these hyperparameters.”
- Also stated in: § Additional information · plain
“In the LM-as-a-service setup, using GPT-Neo, we search for other values for L and k, and note that the choice of our hyperparameters is robust.”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Additional information · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the unsupervised methods, the RANDOM baseline shows that random sampling of training examples leads to poor performance.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the scoring LM and inference LM are identical (using GPT-NEO (Black et al., 2021)), performance compared to the best baseline improves from 26% to 31.9% on BREAK, from 57% to 64.2% on MTOP, and from 51.4% to 54.3% on SMCALFLOW.”
- Also stated in: § Model · plain
“Specifically, when comparing to the best baseline, it improves performance from 26.0 to 31.9 on BREAK, from 57.0 to 64.2 on MTOP, and from 51.4 to 54.3 on SMCALFLOW.”
§ Learning To Retrieve Prompts for In-Context Learning > Similar examples
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Model · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Analogously, when using CODEX as the scoring LM and inference LM performance remains roughly the same: 29.5→29.3.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We evaluate our approach on three sequence-tosequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.”
- Also stated in: § Similar examples · plain
“We observe that EPR substantially improves performance compared to prior work on prompt retrieval.”
§ Learning To Retrieve Prompts for In-Context Learning > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Similar examples · plain
- It may state two separate claims as one. 10
- Probably not a substantial claim. 5
- The quote only partly states this claim. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ANYCORRECT-ORACLE reaches 53.6%, 20 points above LM-ORACLE.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For both, we experimented with passing the retriever the training pair (x, y) or the target sequence y only, and found that using y leads to slightly higher performance.”
§ Learning To Retrieve Prompts for In-Context Learning > 3.1 Generating the Training Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Interestingly, EPR outperforms BM25-ORACLE on MTOP and SMCALFLOW and is comparable on BREAK.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All training examples retrieved by EPR perform an argmax (argmin in the original utterance), and return in the final step”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Zooming in on different inference LMs, GPT-J performs slightly better than GPT-NEO across the board, since it was trained on the same data and using the same procedure as GPT-NEO.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, EPR outperforms the best baseline by 8.5%, and BM25-ORACLE by 5%.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, on MTOP, 84.5% of the examples were abstractly copied, and on that subset of examples, EPR achieves 71.6% EM, compared to 64.2% on the entire validation set.”
§ Learning To Retrieve Prompts for In-Context Learning > Prompt copying
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, GPT-J outperforms CODEX (except on MTOP) and GPT-3 despite being 30x smaller.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Tables 9, 10, and 11 provide more examples for cases where EPR is correct while CBR is incorrect along with the top-3 prompts for each method.”
- Also stated in: § 6 Conclusions · plain
“An example from MTOP development set where EPR is correct and CBR is incorrect along with the top-3 training examples retrieved from each retriever.”
- Also stated in: § 6 Conclusions · plain
“An example from SMCALFLOW development set where EPR is correct and CBR is incorrect along with the top-3 training examples retrieved from each retriever.”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 6 Conclusions · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that the rate of copying is much higher in MTOP and SM-CALFLOW compared to BREAK, where in MTOP and SMCALFLOW abstract copying reaches more than 80%.”
§ Learning To Retrieve Prompts for In-Context Learning > Prompt copying
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Large language models have been shown to exhibit various kinds of bias (Bender et al., 2021)”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
What the sources say
- Not checked yetcites Bender et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“since EPR is trained on the signal obtained from such large LMs, it might also exhibit these biases.”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On BREAK, we evaluate performance on the development set with LF-EM (Hasson and Berant, 2021), which is a better metric compared to Normalized Exact Match (NEM), the official metric, as it measures whether two meaning representations are semantically equivalent.”
§ Learning To Retrieve Prompts for In-Context Learning > 4.3 Experimental Details
What the sources say
- Not checked yetcites Hasson and Berant, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of LM-ORACLE is substantially higher than EPR, showing that the supervision provided by the scoring LM is strong, and training a better retriever from this signal can substantially enhance performance.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“shows that EPR captures both lexical and structure similarity.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
The candidate set Ē includes good prompt candidates and hard negatives.
“This should lead to relevant positive examples, assuming that the set of candidates, Ē includes good prompt candidates and hard negatives, since all candidates have high similarity with (x, y) according to R u (y, D).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although instance-level demonstrations generally perform better than task-level demonstrations, it requires a labeled training set in advance for retrieval.”
From What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs? (2023) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Retrieval allows LLMs to access databases, mitigating hallucinations (Li et al. 2022;Wang, Yang, and Wei 2023;Rubin, Herzig, and Berant 2022;Liu et al. 2022).”
From Expel: LLM agents are experiential learners (2024) · cites this paper as Rubin, Herzig, and Berant 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Recent work has demonstrated that retrieval-based mechanisms can improve language models for various tasks requiring factual or procedural knowledge (Liu et al., 2021;Poesia et al., 2021;Rubin et al., 2022;Madaan et al., 2022;Shrivastava et al., 2023).”
From Learning Performance-Improving Code Edits (2023) · cites this paper as Rubin et al., 2022
In the citing paper
“earlier research has showed several positive pedagogical impacts of using NLP-based social conversational agents on children's perception of their own competencies and their attitudes towards learning ([51]).”
From GPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking Skills (2024) · cites this paper as [51]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Ankerst, M. (1999) · doi:10.1145/304181.304187 · Cited by 3,982
Research works printed but not linked: 2
- Geoffrey E. Hinton and Sam T. Roweis. 2002. Stochas- tic neighbor embedding. In NIPS.
- Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest Neigh- bor Machine Translation. In Proceedings of ICLR.
Cited by
Sorted by citations
Items 301–387 of 387
Retrieval-Enhanced Event Temporal Relation Extraction by Prompt Tuning
Luo, R., & Hu, P. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2421-5_2
How Could Be Used Student Comments for Delivering Feedback to Instructors in Higher Education?
Astudillo, G., Hilliger, I., & Baier, J. A. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64312-5_50
Retrieval-Enhanced Template Generation for Template Extraction
Wang, R., Xiang, W., Wang, Z., & Wang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9431-7_29
SMOOTHIE: Label Free Language Model RoutingPreprint
Guha, N., Chen, M. F., Chow, T., Khare, I. S., & Ré, C. (2025) · Qeios · doi:10.32388/f319w4
Beyond expression: Comprehensive visualization of knowledge triplet facts
Liu, W., He, Y., Wang, C., Xie, S., & Li, W. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104062
Research on the Reasoning Ability of General Language Model Based on Chain of Thought
康, 睿. (2025) · Artificial Intelligence and Robotics Research · doi:10.12677/airr.2025.142026
Logistic Regression-Based Example Selection for Enhanced Few-Shot Learning in Intent Classification
Park, G., & Lee, H. (2025) · International Conference on Consumer Electronics · doi:10.1109/icce63647.2025.10930071
Wu, O. T., Chan, F. K. S., & Law, Y. N. (2024) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait64506.2024.10963264
Few-shot task-oriented dialogue generation with in-context learning
Bi, Z., Hu, X., Zhang, W., & Wang, X. (2025) · International Journal of Web Information Systems · doi:10.1108/ijwis-01-2025-0007
Chen, Y., Hu, G., Xia, C., Yu, L., Popescu, M., & Xu, D. H. (2025) · International Journal of Artificial Intelligence and Robotics Research · doi:10.1142/s2972335325300019 · Cited by 1
Tang, S., He, B., & Yu, X. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11366-4
Formalizing In-Context Learning in Transformers
Chen, P., & Liu, S. (2025) · Springer eBooks · doi:10.1007/978-3-031-76770-8_4
Strengths-Leverage Chain-of-Thought: Enhancing Multimodal Reasoning with LLM and LMM
Xu, Q., & Zhang, L. (2024) · International Conference on Virtual Reality and Visualization · doi:10.1109/icvrv62410.2024.00063
Zhou, J., & Wu, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8725-1_33
Identifying Multi-parameter Constraint Errors in Python Data Science Library API Documentation
Xu, X., Xie, F., Zhu, C., Bai, G., Khurshid, S., & Li, Y. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3728945 · Cited by 1
Improving In-Context Learning with Inquiry Style Classification in Table Question Answering
Liu, W., & Kong, F. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-7008-6_30
Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning
Jain, K., Kate, K., Tsay, J., Goues, C. L., & Hirzel, M. J. (2025) · doi:10.1109/ast66626.2025.00016
Zhang, D., Cheng, H. D., Zhang, J., & Gao, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5380392
Wang, S., Sun, Y., Zhao, X., Jiang, H., Cai, H., Lu, C., & Huang, G. (2025) · IEEE International Conference on Software Services Engineering (SSE) · doi:10.1109/sse67621.2025.00017
General Quality Evaluation Metric Towards Demonstrations for In-Context Learning
Wang, H., Wu, J., Sun, H., Xia, Z., Cheng, D., Wang, J., Qi, Q., & Liao, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3606230
Streamlining Research Complexities for AI Agents: Charting Pathways to Innovative Idea Generation
Guo, J., Akbar, K. A., Khan, L. R., Thuraisingham, B., & Irtiza, S. (2025) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa65407.2025.11166476
Rethinking Data Use in Large Language Models
Min, S., Hajishirzi, H., & Zettlemoyer, L. (2025) · Computational Linguistics · doi:10.1162/coli.a.573
How to quickly select good in-context examples in large language models for data-to-text tasks?
Li, Y., Yang, J., Jiang, L., Liu, S., & An, N. (2025) · Natural language processing. · doi:10.1017/nlp.2025.10010
Ni, X., Zhao, Q., & Huang, S. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0329
Wang, Y., Zhang, Z., Shan, L., & Liu, B. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3349-7_45
Hear ALL Parties: Detecting Implicit Hate Speech using Contrastive In-context Learning
Li, Y., Guo, S., & Zhang, X. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228739
RAR: A Refined-Attitude Reasoning Framework on LLMs for Zero Shot Stance Detection
Lee, Y., Zhu, Y., Chen, X., & Wu, B. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11229139
Unsupervised Text Style Transfer via LLMs and Mask-Filling with Multi-way Interactions
Pan, L., Liang, Y., Xu, H., Ma, D., Su, Z., & Fan, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4367-0_14
Effects of Image Samples on In-Context Learning of Multimodal Large Language Models
Ikeda, T., & Yamamoto, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11976-6_35
DEDALUS & ICARUS: Image Privacy Classification Systems with Risk Oriented Explanations
Alba, H. R. D., Aı̈meur, E., Loutis, M., & Alqahtani, K. (2025) · Conference on Privacy, Security and Trust · doi:10.1109/pst65910.2025.11268869
Comparison of representative demonstration designing methods of Language Models
Tiwari, S. (2023) · Open Research Knowledge Graph (ORKG) · doi:10.48366/r598308
Adversarial distractor generation for MCQA: Leveraging in-context learning and rule-based approaches
Yiğit, G., & Amasyalı, M. F. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100186
Retrieval-Augmented Prompt Learning for Pre-Trained Foundation Models
Chen, X., Ou, Y., Feng, Q., Li, L., Li, P., Ye, H., Huang, S.-J., Qiao, S., Deng, S., Chen, H., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3608936
Towards a Robust RAG System for Personalized Learning
Nguyen, V. D., & Debnath, N. C. (2025) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-07326-6_19
Falcon: A Universal Text-Only Membership Inference Attack Framework Against In-Context Learning
Su, H., Qin, Y., Li, Z., Miao, X., & Zhou, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3640876
Unlocking In-Context Learning for Natural Datasets Beyond Language Modelling
Bratulić, J., Mittal, S., Hoffmann, D. T., Böhm, S., Schirrmeister, R. T., Ball, T., Rupprecht, C., & Brox, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-12840-9_20 · Cited by 2
Training an AI Chatbot to Manage Health in Underserved Populations: Methodological Approach
Crawford, A. D., Wicks, B., Metsis, V., Starfall, A., Clapham, F., Gorbachev, A., Shanley, S., Strauser, C., & McGrath, J. M. (2026) · JMIR AI · doi:10.2196/84145
Bao, M., Zhang, K., Liu, X., Liu, Q., Zhao, H., & Chen, E. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3658068
Teacher-Guided Code Generation With API Update
Sun, T., Wang, Z., Wang, W., Chen, R., & Pei, J. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342663
Large Language Model Judged Self-Training for Named Entity Recognition
Chen, S., Wang, J., Yang, C., Xiao, Y., Li, Z., & Lin, X. A. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3778009
Zero-Query Order-Aware Exemplar Selection with a Coherence-Relevance Metric
Guo, X., & Chen, Z. (2025) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice68195.2025.11382278
Improving Cloze Distractor Generation Through Retrieval-based Example Selection
Dong, J., Bai, J., Zhang, J., Li, C., Lin, X., & Rong, W. (2025) · IEEE Smart World Congress (SWC) · doi:10.1109/swc65939.2025.00193
Nanda, S., Majee, A., & Iyer, R. (2025) · Proceedings · doi:10.1109/icdm65498.2025.00154
Guo, Y., Zhou, Y., Hu, D., Sutari, S., Chow, E., Tam, S. P., Perret, D., Pandita, D., & Zheng, K. (2026) · medRxiv · doi:10.64898/2026.02.27.26347290
Chain-of-Thought Construction and Example Optimization in Prompt Learning for Power Customer Service
Xu, H., Shi, C., Liu, T., Zheng, H., & Zhang, Y. (2025) · International Conference on Power Engineering (ICPE) · doi:10.1109/icpe68635.2025.11407689
Joshi, S., & Jadhav, K. S. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00420
Implicit Demonstration Augmentation for Robust and Stable In-Context Learning
Zhou, X., Lee, Z., Xie, R., Zou, L., Ye, W., & Zhang, S. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3677885
Enhanced Schema Linking with Large Language Models via Self-Verification and Value Hints
Ma, L., Wei, D., Li, X., Wen, F., & Zhang, H. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10040104
Wang, Z., Liu, R., & Fu, J. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_5
In-Context Learning in Large Language Models: Mechanisms, Challenges, and Frontiers
Yuan, Y. (2026) · ITM Web of Conferences · doi:10.1051/itmconf/20268403003 · Cited by 1
The Example Saturation Effect: The Hidden Role of Input Difficulty in In-Context Learning
Fan, X., Peng, B., Sun, H., Guo, J., Yang, C., & Zhang, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463529
Reward-Based Efficient Demonstration Selection for In-Context Learning
Liang, Y., Ding, N., Wan, M., Liu, M.-Y., & Dong, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461481
Li, Y., Wu, T., Gao, J., Zhu, W., & Ouyang, W. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462821
Zhao, X., Hu, R., Li, X., Qiu, D., Wang, Y., Yu, J., Liu, S., & Sang, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464276
DualAST: AST-Guided Exemplar Retrieval for In-Context Learning in Multi-Step Reasoning
Huang, H., Liu, C., & Yuan, M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462067
Jung, W. J., Kwak, N.-J., & Lee, B.-Y. (2026) · Applied Sciences · doi:10.3390/app16094100
ICL-Agent: Improving LLM Agent Capabilities via a Novel In-Context Learning Approach
Xie, C., Sun, Y. W., & Hu, B. (2026) · International Conference on Electronics Technology and Artificial Intelligence (ETAI) · doi:10.1109/etai68332.2026.11485338
T2LF: LLM-Guided Multimodal Diffusion for Text-to-Light Field Synthesis
Yoon, S., Ahn, N., & Park, I. K. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00707
He, B., Du, Y., Meng, H., Tang, S., Zhang, T., & Song, L. (2026) · Symmetry · doi:10.3390/sym18050792 · Cited by 1
Precedent Retrieval Based In-Context Learning for Legal Rationale Generation
Yue, L., Du, Y., & Gao, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0378-9_8
Sun, X., Li, M., Zhao, J., & Liu, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0378-9_4
A Context-Aware Reinforcement Learning Framework for Adaptive Multimodal Prompt OptimizationPreprint
Shashank, R., Shreeshanth, R., Rajagopal, M., Karthi, G., & Rajakrishnammal, A. (2026) · Research Square · doi:10.21203/rs.3.rs-9637361/v1
Calibrating In-Context Bias in Large Language Model Predictions for Event Causality Identification
Xiang, W., Zhan, C., Du, X., & Wang, B. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3692294
Dang, L. H., Le, T. M., Le, V., Phuong, T. M., & Tran, T. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01411-z · Cited by 2
Anna, N. E. V., Chansanam, W., Mannan, E. F., Rahmah, E., & Alshafa, Y. N. (2026) · Journal of Library Administration · doi:10.1080/01930826.2026.2662298
Yurosko, R., Hevner, A. R., Jank, W., & Zantedeschi, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-28316-0_11
Jing, G., Gao, P. F., Lee, Y., & Zhang, H. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810768
Unlocking In-Context Learning for Natural Datasets Across Modalities
Bratulić, J., Mittal, S., Hoffmann, D. T., Böhm, S., Schirrmeister, R. T., Ball, T., Rupprecht, C., & Brox, T. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02913-0
Optimizing In-Context Demonstrations for LLM-Based Automated Grading
Chu, Y., Li, H., Yang, K., Copur‐Gencturk, Y., Haudek, K. C., Krajcik, J., Shin, N., & Tang, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29755-6_20
Yang, X. (2026) · International Conference on Artificial Intelligence and Industrial Technology Applications (AIITA) · doi:10.1109/aiita69518.2026.11567207
Yang, X., & Li, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3400-4_3
Does Representational Instruction Tuning Improve Conditional Sentence Representations?
Krithiwas, G., Gautam, J., & Mamatha, H. R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-24807-7_2
Transformer-Based In-Context Learning for Distributed Channel Access Optimization
Hao, S., Li, H., & Duan, L. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3713842
Luo, S., Xue, M., Zeng, L., Ouyang, T., Wang, P., Guo, S., & Chen, X. (2026) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/2575-8411.2026.00129
Dynamic Reliable-Aware Router for Few-Shot Multimodal Sentiment Analysis
Lin, J., Cai, T., Chen, W., Yang, Z., & Liao, X. (2026) · IEEE International Conference on Computer Vision and Deep Learning (DLCV) · doi:10.1109/dlcv69906.2026.11635125
Cao, S. (2026) · International Conference on Artificial Intelligence and Electromechanical Automation (AIEA) · doi:10.1109/aiea70743.2026.11632893
Ray, K., & Tripathi, R. (2026) · IEEE Guwahati Subsection Conference (GCON) · doi:10.1109/gcon69192.2026.11648277
Generative AI Technologies, Techniques and Tensions: A Primer
Behrens, J. T. (2026) · Springer eBooks · doi:10.1007/978-3-032-24819-0_2 · Cited by 2
Quality-aware Scoring and Filtering of Sentence Analogies for Analogical Reasoning
Qin, Z., Song, P., & Sun, S. (2026) · International Conference on Machine Learning and Neural Networks · doi:10.1145/3821725.3821818
ReVerTree: Recursive Verification Tree for Self-checking Mathematical Reasoning
You, C., Zhang, T., Chen, Q., Wang, C., & He, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-5696-9_14
Few-Shot Learning for Low-Resource Natural Language Inference for French
Dievart, C., Ammari, R., Aljibbaoui, T., Skandalis, M., & Sioutis, M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-6513-8_9
Thomo, T., & Thomo, A. (2026) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-026-00883-4
Liu, H., Zhang, Q., Marcus, R., & Sabek, I. (2026) · The VLDB Journal · doi:10.1007/s00778-026-00999-z
Beyond Semantic Similarity: Multi-Stage Calibration for In-Context Learning
Xi, C., & Liang, D. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729913
Bayesian Demonstration Selection for LLM-Based Dementia Detection From Patient-Generated Text
Rahman, M. A., Babiker, H. K. B., Hossain, M. R. I., Kim, M., & Goebel, R. G. (2026) · IEEE Access · doi:10.1109/access.2026.3744279
Li, D., Kong, D., 吴怀光, Jia, W., Qian, S., & Cai, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134835
From Scene to Glyph: Hierarchical Retrieval for Multi-Modal In-Context Scene Text Recognition
Hu, Z., Jiang, Y., & Zhong, D. (2026) · Electronics · doi:10.3390/electronics15214969
Metadata
- Authors
- Rubin, O., Herzig, J., & Berant, J.
- Affiliations
- Tel Aviv University (IL)
- Venue
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2022
- Pages
- 2655-2671
- OpenAlex
- W4287891464
- Semantic Scholar
- f9838a3be5c94bb2674a0e224de349b50e18f3c4
- Cited by
- 374
- References
- 54
- Citations by year
- 2026: 57 · 2025: 97 · 2024: 91 · 2023: 119 · 2022: 10
- FWCI
- 43.96
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- in-context learning, dense retrievers, training data selection, pre-trained language models, annotated dataset, meaning representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
