The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index; none of the 3 sources its claims cite is checked for support yet.
Learning To Retrieve Prompts for In-Context Learning
Open accessAbstract
In-context learning is a recent paradigm in natural language understanding, where a large pretrained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters.However, performance has been shown to strongly depend on the selected training examples (termed prompts).In this work, we propose an efficient method for retrieving prompts for in-context learning using annotated data and an LM.Given an inputoutput pair, we estimate the probability of the output given the input and a candidate training example as the prompt, and label training examples as positive or negative based on this probability.We then train an efficient dense retriever from this data, which is used to retrieve training examples as prompts at test time.We evaluate our approach on three sequence-tosequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper proposes Efficient Prompt Retrieval (EPR), which uses a language model to label which training examples are good in-context prompts and trains a dense retriever from that signal. EPR substantially outperforms prior prompt-retrieval baselines on BREAK, MTOP, and SMCALFLOW.
Key findings
- With GPT-NEO as both scoring and inference LM, EPR improves over the best baseline from 26.0 to 31.9 on BREAK, from 57.0 to 64.2 on MTOP, and from 51.4 to 54.3 on SMCALFLOW.
- In the one-shot BREAK setting, EPR reaches 23.0 LF-EM, outperforming the best baseline by 8.5 points and BM25-ORACLE by 5 points.
- When GPT-NEO is used as a proxy scoring LM for larger inference LMs, EPR outperforms BM25 and CBR by 2–8 points across all datasets and pretrained models.
- Prompt-copying analysis shows high abstract copying in MTOP and SMCALFLOW (>80%); on MTOP, 84.5% of examples were abstractly copied, and EPR achieved 71.6% EM on that copied subset versus 64.2% overall.
Methods
- EPR generates training data by retrieving candidate prompts with an unsupervised retriever (BM25 or SBERT; BM25 chosen), scoring each candidate with a scoring LM as Prob(y | candidate, x), and labeling top-k as positive and bottom-k as negative (k=5, L=50).
- It trains a dense retriever with DPR-style contrastive learning using BERT-base input and prompt encoders, then uses FAISS maximum inner-product search at inference to retrieve prompts within the inference LM’s context limit.
- Evaluation covers three structured sequence-to-sequence tasks—BREAK, MTOP, and SMCALFLOW—using GPT-NEO as scoring LM and GPT-NEO, GPT-J, GPT-3, or CODEX as inference LMs, with LF-EM/NEM/EM metrics and a 2,048-token context size.
Limitations
- Paper-stated: the authors note that large language models exhibit bias, and since EPR is trained on signal from such LMs, it might also exhibit these biases.
- Paper-stated: the method scores each training example independently at training time, while at test time the LM observes a sequence of examples; modeling dependence between examples is left to future work.
- Evident from the text: evaluation is limited to three English structured tasks mapping utterances to meaning representations, and the approach relies on a scoring LM as a proxy for the inference LM.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
19 Own result · 11 Own interpretation · 6 No basis given · 3 Not checked yet
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows test results on BREAK and MTOP corroborating that EPR substantially improves performance compared to BM25 and CBR.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“BM25 outperforms SBERT for prompt retrieval, and consequently we use BM25 in all of our supervised approaches to retrieve the set of candidates, Ē.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“EPR substantially outperforms all other baselines.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The candidate set Ē includes good prompt candidates and hard negatives.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“CODEX outperforms GPT-3, which can be explained by the fact that it was trained on code, and our datasets involve mapping to programs or meaning representations.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, accuracy on examples where copying occurred is much higher compared to accuracy where no copying happened.”
§ Learning To Retrieve Prompts for In-Context Learning > Prompt copying
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We test the effect of k, the number of prompts labeled as positive or negative, and L, the number of prompts retrieved by the unsupervised retriever.Table 8 shows that performance is is generally robust w.r.t these hyperparameters.”
- Also stated in: § Additional information · plain
“In the LM-as-a-service setup, using GPT-Neo, we search for other values for L and k, and note that the choice of our hyperparameters is robust.”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Additional information · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the unsupervised methods, the RANDOM baseline shows that random sampling of training examples leads to poor performance.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the scoring LM and inference LM are identical (using GPT-NEO (Black et al., 2021)), performance compared to the best baseline improves from 26% to 31.9% on BREAK, from 57% to 64.2% on MTOP, and from 51.4% to 54.3% on SMCALFLOW.”
- Also stated in: § Model · plain
“Specifically, when comparing to the best baseline, it improves performance from 26.0 to 31.9 on BREAK, from 57.0 to 64.2 on MTOP, and from 51.4 to 54.3 on SMCALFLOW.”
§ Learning To Retrieve Prompts for In-Context Learning > Similar examples
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Model · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Analogously, when using CODEX as the scoring LM and inference LM performance remains roughly the same: 29.5→29.3.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We evaluate our approach on three sequence-tosequence tasks where language utterances are mapped to meaning representations, and find that it substantially outperforms prior work and multiple baselines across the board.”
- Also stated in: § Similar examples · plain
“We observe that EPR substantially improves performance compared to prior work on prompt retrieval.”
§ Learning To Retrieve Prompts for In-Context Learning > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Similar examples · plain
- It may state two separate claims as one. 10
- Probably not a substantial claim. 5
- The quote only partly states this claim. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ANYCORRECT-ORACLE reaches 53.6%, 20 points above LM-ORACLE.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For both, we experimented with passing the retriever the training pair (x, y) or the target sequence y only, and found that using y leads to slightly higher performance.”
§ Learning To Retrieve Prompts for In-Context Learning > 3.1 Generating the Training Data
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Interestingly, EPR outperforms BM25-ORACLE on MTOP and SMCALFLOW and is comparable on BREAK.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All training examples retrieved by EPR perform an argmax (argmin in the original utterance), and return in the final step”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Zooming in on different inference LMs, GPT-J performs slightly better than GPT-NEO across the board, since it was trained on the same data and using the same procedure as GPT-NEO.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, EPR outperforms the best baseline by 8.5%, and BM25-ORACLE by 5%.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, on MTOP, 84.5% of the examples were abstractly copied, and on that subset of examples, EPR achieves 71.6% EM, compared to 64.2% on the entire validation set.”
§ Learning To Retrieve Prompts for In-Context Learning > Prompt copying
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, GPT-J outperforms CODEX (except on MTOP) and GPT-3 despite being 30x smaller.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Tables 9, 10, and 11 provide more examples for cases where EPR is correct while CBR is incorrect along with the top-3 prompts for each method.”
- Also stated in: § 6 Conclusions · plain
“An example from MTOP development set where EPR is correct and CBR is incorrect along with the top-3 training examples retrieved from each retriever.”
- Also stated in: § 6 Conclusions · plain
“An example from SMCALFLOW development set where EPR is correct and CBR is incorrect along with the top-3 training examples retrieved from each retriever.”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 6 Conclusions · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that the rate of copying is much higher in MTOP and SM-CALFLOW compared to BREAK, where in MTOP and SMCALFLOW abstract copying reaches more than 80%.”
§ Learning To Retrieve Prompts for In-Context Learning > Prompt copying
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Large language models have been shown to exhibit various kinds of bias (Bender et al., 2021)”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
What the sources say
- Not checked yetcites Bender et al., 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“since EPR is trained on the signal obtained from such large LMs, it might also exhibit these biases.”
§ Learning To Retrieve Prompts for In-Context Learning > Additional information
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On BREAK, we evaluate performance on the development set with LF-EM (Hasson and Berant, 2021), which is a better metric compared to Normalized Exact Match (NEM), the official metric, as it measures whether two meaning representations are semantically equivalent.”
§ Learning To Retrieve Prompts for In-Context Learning > 4.3 Experimental Details
What the sources say
- Not checked yetcites Hasson and Berant, 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of LM-ORACLE is substantially higher than EPR, showing that the supervision provided by the scoring LM is strong, and training a better retriever from this signal can substantially enhance performance.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“shows that EPR captures both lexical and structure similarity.”
§ Learning To Retrieve Prompts for In-Context Learning > Model
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
The candidate set Ē includes good prompt candidates and hard negatives.
“This should lead to relevant positive examples, assuming that the set of candidates, Ē includes good prompt candidates and hard negatives, since all candidates have high similarity with (x, y) according to R u (y, D).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although instance-level demonstrations generally perform better than task-level demonstrations, it requires a labeled training set in advance for retrieval.”
From What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs? (2023) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Retrieval allows LLMs to access databases, mitigating hallucinations (Li et al. 2022;Wang, Yang, and Wei 2023;Rubin, Herzig, and Berant 2022;Liu et al. 2022).”
From Expel: LLM agents are experiential learners (2024) · cites this paper as Rubin, Herzig, and Berant 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Recent work has demonstrated that retrieval-based mechanisms can improve language models for various tasks requiring factual or procedural knowledge (Liu et al., 2021;Poesia et al., 2021;Rubin et al., 2022;Madaan et al., 2022;Shrivastava et al., 2023).”
From Learning Performance-Improving Code Edits (2023) · cites this paper as Rubin et al., 2022
In the citing paper
“earlier research has showed several positive pedagogical impacts of using NLP-based social conversational agents on children's perception of their own competencies and their attitudes towards learning ([51]).”
From GPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking Skills (2024) · cites this paper as [51]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Ankerst, M. (1999) · doi:10.1145/304181.304187 · Cited by 3,982
Research works printed but not linked: 2
- Geoffrey E. Hinton and Sam T. Roweis. 2002. Stochas- tic neighbor embedding. In NIPS.
- Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest Neigh- bor Machine Translation. In Proceedings of ICLR.
Cited by
Sorted by citations
Items 101–200 of 387
Selective Demonstrations for Cross-domain Text-to-SQL
Chang, S., & Fosler‐Lussier, E. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.944 · Cited by 30
SummQA at MEDIQA-Chat 2023: In-Context Learning with GPT-4 for Medical Summarization
Mathur, Y., Rangreji, S., Kapoor, R., Palavalli, M., Bertsch, A., & Gormley, M. R. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.51 · Cited by 22
Software Vulnerability Detection with GPT and In-Context Learning
Liu, Z., Liao, Q., Gu, W., & Gao, C. (2023) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc59305.2023.00041 · Cited by 22
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1416-x · Cited by 18
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 143
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.07616 · Cited by 18
Vithanage, D., Deng, C., Wang, L., Yin, M., Alkhalaf, M., Zhang, Z., Zhu, Y., & Yu, P. (2025) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-025-00190-z · Cited by 20
prompt4vis: prompting large language models with example mining for tabular data visualization
Li, S., Chen, X., Song, Y., Song, Y., Zhang, C., Hao, F., & Chen, L. (2025) · The VLDB Journal · doi:10.1007/s00778-025-00912-0 · Cited by 22
FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models
Li, Y., Liu, X., Wang, X., Lee, B., Wang, S., Rocha, A. D. R., & Lin, W. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3597211 · Cited by 59
Compromising LLM Driven Embodied Agents With Contextual Backdoor Attacks
Liu, A., Zhou, Y., Liu, X., Zhang, T., Liang, S., Wang, J., Pu, Y., Li, T., Zhang, J., Zhou, W., et al. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3555410 · Cited by 28
Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap
Huang, K., Wang, F., Huang, Y., & Arora, C. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00081 · Cited by 32
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Yu, Z., Xiong, C., Shi, Y., & Liu, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.136 · Cited by 65
Prompting GPT-3.5 for Text-to-SQL with De-semanticization and Skeleton Retrieval
Guo, C., Tian, Z., Tang, J., Wang, P., Wen, Z., Yang, K., & Wang, T. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7022-3_23 · Cited by 35
Making Large Language Models Better Data Creators
Lee, D., Pujara, J., Sewak, M., White, R. W., & Jauhar, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.948 · Cited by 28
Prompting Large Language Models for Malicious Webpage Detection
Li, L., & Gong, B. (2023) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml59573.2023.10348229 · Cited by 19
Instruct Me More! Random Prompting for Visual In-Context Learning
Zhang, J., Wang, B., Li, L., Nakashima, Y., & Nagahara, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00258 · Cited by 20
Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning
Wang, X., Fang, Z., Li, X., Li, X., Chen, C., & Liu, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00236 · Cited by 20
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Yu, Z., Xiong, C., Yu, S., & Liu, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.17331 · Cited by 17
Reinforced Prompt Personalization for Recommendation with Large Language Models
Mao, W., Wu, J., Chen, W., Gao, C., Wang, X., & He, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3716320 · Cited by 20
Zou, G., Li, P., Yang, S., Hu, S., Pang, S., & Gan, Y. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104219 · Cited by 19
DREAM: Improving Situational QA by First Elaborating the Situation
Gu, Y., Dalvi, B. B., & Clark, P. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.82 · Cited by 20
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations
Lyu, X., Min, S., Beltagy, I., Zettlemoyer, L., & Hajishirzi, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.129 · Cited by 53
Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning
Long, Q., Wang, W., & Pan, S. J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.402 · Cited by 25
Distilling mathematical reasoning capabilities into Small Language Models
Zhu, X., Li, J., Liu, Y., Ma, C., & Wang, W. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106594 · Cited by 29
Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking
King, B., & Flanigan, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.344 · Cited by 22
Data Curation Alone Can Stabilize In-context Learning
Chang, T.-Y., & Jia, R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.452 · Cited by 37
Using In-Context Learning to Improve Dialogue Safety
Meade, N., Gella, S., Hazarika, D., Gupta, P., Jin, D., Reddy, S., Liu, Y., & Hakkani-Tur, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.796 · Cited by 13
Tan, X., Shi, S., Qiu, X., Qu, C., Qi, Z., Xu, Y., & Qi, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.62 · Cited by 22
Dietrich, J., & Hollstein, A. (2024) · Drug Safety · doi:10.1007/s40264-024-01499-1 · Cited by 22
CoLE: A collaborative legal expert prompting framework for large language models in law
Li, B., Fan, S., Zhu, S., & Wen, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113052 · Cited by 15
PaRaDe: Passage Ranking using Demonstrations with LLMs
Drozdov, A., Zhuang, H., Dai, Z., Qin, Z., Rahimi, R., Wang, X., Alon, D., Iyyer, M., McCallum, A., Metzler, D., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.950 · Cited by 15
impact of sample selection on in-context learning for entity extraction from scientific writing
Bölücü, N., Rybinski, M., & Wan, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.338 · Cited by 16
Xu, S., Liu, S., Culhane, T., Pertseva, E., Wu, M.-H., Semnani, S. J., & Lam, M. S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.353 · Cited by 26
Comparative Analysis of Large Language Models in Chinese Medical Named Entity Recognition
Zhu, Z., Zhao, Q. Z., Li, J., Ge, Y., Ding, X., Gu, T., Zou, J., Lv, S., Wang, S., & Yang, J. (2024) · Bioengineering · doi:10.3390/bioengineering11100982 · Cited by 14
Adapting Vision-Language Models via Learning to Inject Knowledge
Xuan, S., Yang, M., & Zhang, S. S. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3468884 · Cited by 15
Sun, J., Pan, Y., & Yan, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129219 · Cited by 17
Refactoring for Dockerfile Quality: A Dive into Developer Practices and Automation Potential
Ksontini, E., Mastouri, M., Khalsi, R., & Kessentini, W. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00116 · Cited by 17
Bansal, H., Gopalakrishnan, K., Dingliwal, S., Bodapati, S., Kirchhoff, K., & Roth, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.660 · Cited by 47
Annotating Materials Science Text: A Semi-automated Approach for Crafting Outputs with Gemini Pro
Sayeed, H. M., Mohanty, T., & Sparks, T. D. (2024) · Integrating materials and manufacturing innovation · doi:10.1007/s40192-024-00356-4 · Cited by 13
Saleem, S., Asim, M. N., van Elst, L., & Dengel, A. R. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01707-6 · Cited by 13
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models
Zhang, Z., Wang, S., Yu, W., Yi‐chong, X., Iter, D., Zeng, Q., Liu, Y., Zhu, C., & Jiang, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.659 · Cited by 22
How Many Demonstrations Do You Need for In-context Learning?
Chen, J., Chen, L., Zhu, C., & Zhou, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.745 · Cited by 12
Pecher, B., Srba, I., & Bieliková, M. (2024) · ACM Computing Surveys · doi:10.1145/3691339 · Cited by 19
Is Relevance Propagated from Retriever to Generator in RAG?
Tian, F., Ganguly, D., & Macdonald, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_3 · Cited by 14
Li, S., Chen, J., Yuan, S., Wu, X., Yang, H., Tao, S., & Xiao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29817 · Cited by 32
Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval
Rubin, O., & Berant, J. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00693 · Cited by 38
Self-citationGuo, C., Tian, Z., Tang, J., Li, S., & Wang, T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103978 · Cited by 15
Mounsif, M., & Medard, F. (2023) · Energy and AI · doi:10.1016/j.egyai.2023.100242 · Cited by 10
Prompt-Augmented Linear Probing: Scaling beyond the Limit of Few-Shot In-Context Learners
Cho, H., Kim, H. J., Kim, J., Lee, S.-W., Lee, S., Yoo, K. M., & Kim, T. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26495 · Cited by 14
LLM Instruction-Example Adaptive Prompting (LEAP) Framework for Clinical Relation ExtractionPreprint
Zhou, H., Li, M., Xiao, Y., Yang, H. Q., & Zhang, R. (2023) · medRxiv · doi:10.1101/2023.12.15.23300059 · Cited by 11
RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction
Meng, S., Hu, X., Liu, A., Li, S., Ma, F., Yang, Y., & Wen, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.316 · Cited by 16
Khanshan, A., Gorp, P. V., & Markopoulos, P. (2024) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3661143 · Cited by 12
Rethinking and Improving Visual Prompt Selection for In-Context Learning Segmentation
Suo, W., Lai, L., Sun, M., Zhang, H., Wang, P., & Zhang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_2 · Cited by 13
Working Memory Capacity of ChatGPT: An Empirical Study
Gong, D., Wan, X., & Wang, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28868 · Cited by 19
Inferring Implicit Relations in Complex Questions with Language Models
Katz, U., Geva, M., & Berant, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.188 · Cited by 13
Self-citationHuang, Q. Y., Sun, Y., Xing, Z., Cao, Y., Chen, J., Xu, X., Jin, H., & Lu, J. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3680469 · Cited by 9
Liu, Y., Zhu, J., Liu, X., Tang, H., Zhang, Y., Zhang, K., Zhou, X., & Chen, E. (2025) · Proceedings · doi:10.1109/icdm65498.2025.00055 · Cited by 23
LLM With Relation Classifier for Document-Level Relation Extraction
Li, X., Chen, K., Long, Y., & Zhang, M. (2026) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2026.3668580 · Cited by 34
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple-Choice Perspective
Wang, J., Yang, P., Gan, R., Zhang, Y., Zhang, J., & Sakai, T. (2023) · IEEE Access · doi:10.1109/access.2023.3343123 · Cited by 10
What Makes Pre-trained Language Models Better Zero-shot Learners?
Lu, J., Zhu, D., Han, W., Zhao, R., Namee, B. M., & Tan, F. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.128 · Cited by 17
Bootstrapping Multilingual Semantic Parsers using Large Language Models
Awasthi, A., Gupta, N., Samanta, B., Dave, S., Sarawagi, S., & Talukdar, P. P. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.180 · Cited by 12
Meta In-Context Learning: Harnessing Large Language Models for Electrical Data Classification
Zhou, M., Li, F., Zhang, F., Zheng, J., & Ma, Q. (2023) · Energies · doi:10.3390/en16186679 · Cited by 12
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations
Chen, W., Wu, C.-K., Chen, Y.-N., & Chen, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.968 · Cited by 33
Zhou, H., Wan, X., Vulić, I., & Korhonen, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.870 · Cited by 17
Evaluating the Effectiveness and Efficiency of Demonstration Retrievers in RAG for Coding Tasks
He, P., Wang, S., Chowdhury, S., & Chen, T. H. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00053 · Cited by 16
Understanding Human-AI Misalignment in LLM-Based Job-Seeking Support for Neurodivergent Users
Hall, K., Ma, M., Zhang, X., Swain, V. D., & Kim, J. G. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746361 · Cited by 8
Robust Prompt Optimization for Large Language Models Against Distribution Shifts
Li, M., Wang, W., Feng, F., Cao, Y., Zhang, J., & Chua, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.95 · Cited by 19
DG-PIC: Domain Generalized Point-In-Context Learning for Point Cloud Understanding
Jiang, J., Zhou, Q., Li, Y., Lu, X., Wang, M., Ma, L., Chang, J. M., & Zhang, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_26 · Cited by 8
Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning
Jie, Z., & Lu, W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.668 · Cited by 15
Model-tuning Via Prompts Makes NLP Models Adversarially Robust
Raman, M., Maini, P., Kolter, J. Z., Lipton, Z. C., & Pruthi, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.576 · Cited by 10
A Prompt Example Construction Method Based on Clustering and Semantic Similarity
Chen, D., & Wang, J. (2024) · Systems · doi:10.3390/systems12100410 · Cited by 9
Metric-Based In-context Learning: A Case Study in Text Simplification
Vadlamannati, S., & Şahin, G. G. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.18 · Cited by 8
PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching
Qi, Z., Tan, X., Shi, S., Qu, C. Y., Xu, Y., & Qi, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.45 · Cited by 9
Efficient In-Context Medical Segmentation with Meta-Driven Visual Prompt Selection
Wu, C., Restrepo, D. S., Shuai, Z., Liu, Z., & Shen, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72114-4_25 · Cited by 6
Zhang, Y., Huang, B., Yuan, W., Jiang, Z., Peng, L., Chen, S., & Tan‐Soo, J. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104090 · Cited by 7
Valuing Training Data via Causal Inference for In-Context Learning
Zhou, X., Ye, W., Lee, Z., Zou, L., & Zhang, S. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3546761 · Cited by 7
One Size Doesn’t Fit All: Predicting the Number of Examples for In-Context Learning
Chandra, M., Ganguly, D., & Ounis, I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_5 · Cited by 12
Universal Self-Adaptive Prompting
Wan, X., Sun, R., Nakhost, H., Dai, H., Eisenschlos, J. M., Arık, S. Ö., & Pfister, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.461 · Cited by 21
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting
Ye, X., & Durrett, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.41 · Cited by 11
Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning
Liu, H., & Wang, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1060 · Cited by 10
SINC: Self-Supervised In-Context Learning for Vision-Language Tasks
Chen, Y.-S., Song, Y.-Z., Yeo, C. Y., Liu, B., Fu, J., & Shuai, H.-H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01415 · Cited by 7
Chen, Y., Xi, N., Du, Y., Wang, H., Chen, J., Zhao, S., & Qin, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30198 · Cited by 11
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis
Yang, S., Kim, J., Jang, J., Ye, S., Lee, H., & Seo, M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00666 · Cited by 12
Zero-Shot Prompting for LLM-Based Machine Translation Using In-Domain Target Sentences
Ji, B., Duan, X., Zhang, Y., Wu, K., & Zhang, M. (2024) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3519814 · Cited by 9
Mimic In-Context Learning for Multimodal Tasks
Jiang, Y.-C., Fu, J., Hao, C.-D., Hu, X., Peng, Y., Geng, X., & Yang, X. (2025) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52734.2025.02776 · Cited by 21
IDAS: Intent Discovery with Abstractive Summarization
Raedt, M. D., Godin, F., Demeester, T., & Develder, C. (2023) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2023.nlp4convai-1.7 · Cited by 21
Zhao, Y., Zhang, Y., Sui, D., Cao, L., Zhao, J., & Liu, K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.331 · Cited by 13
TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision
Zhou, R., Yang, Y., Wen, M., Wen, Y., Wang, W., Xi, C., Xu, G., Yu, Y. H., & Zhang, W. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657788 · Cited by 18
Task Navigator: Decomposing Complex Tasks for Multimodal Large Language Models
Ma, F., Zhou, Y., Zhang, Y., Wu, S., Zhang, Z., He, Z., Rao, F., & Sun, X. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00230 · Cited by 6
M2R-Whisper: Multi-stage and Multi-scale Retrieval Augmentation for Enhancing Whisper
Zhou, J., Zhao, S., He, J., Wang, H., Zeng, W., Chen, Y., Sun, H., Kong, A., & Qin, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888086 · Cited by 6
Query Refinement Prompts for Closed-Book Long-Form QA
Amplayo, R. K., Webster, K., Collins, M., Das, D., & Narayan, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.444 · Cited by 23
Kang, D., Park, J., Jo, Y., & Bak, J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.961 · Cited by 17
Semantic Verification in Large Language Model-based Retrieval Augmented Generation
Martin, A., Witschel, H. F., Mandl, M. M., & Stockhecke, M. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31199 · Cited by 7
Complex QA and language models hybrid architectures, SurveyPreprint
Daull, X., Bellot, P., Bruno, E., Martin, V., & Murisasco, É. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09051 · Cited by 32
Grimoire is All You Need for Enhancing Large Language ModelsPreprint
Chen, D., Song, S., Yu, Q., Li, Z., Wang, W., Xiong, F., & Tang, B. (2024) · Research Square · doi:10.21203/rs.3.rs-3845612/v1 · Cited by 13
Vithanage, D., Deng, C., Wang, L., Yin, M., Alkhalaf, M., Zhang, Z., Zhu, Y., Soewargo, A. C., & Yu, P. (2024) · medRxiv · doi:10.1101/2024.06.24.24309441 · Cited by 6
Zha, Y., Li, Y., & Lu, X. (2024) · Mathematics · doi:10.3390/math12193141 · Cited by 5
Gilani, K., Verket, M., Peters, C., Dumontier, M., Rocca, H.-P. B.-L., & Urovi, V. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110745 · Cited by 5
Chopra, A., & Azam, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-62495-7_12 · Cited by 5
WebGLM: Towards an Efficient and Reliable Web-Enhanced Question-Answering System
Lai, H., Liu, X., Yu, H., Xu, Y., Iong, I. L., Yao, S., Zeng, A., Du, Z., Dong, Y., & Tang, J. (2025) · ACM Transactions on Information Systems · doi:10.1145/3729421 · Cited by 7
Metadata
- Authors
- Rubin, O., Herzig, J., & Berant, J.
- Affiliations
- Tel Aviv University (IL)
- Venue
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2022
- Pages
- 2655-2671
- OpenAlex
- W4287891464
- Semantic Scholar
- f9838a3be5c94bb2674a0e224de349b50e18f3c4
- Cited by
- 374
- References
- 54
- Citations by year
- 2026: 57 · 2025: 97 · 2024: 91 · 2023: 119 · 2022: 10
- FWCI
- 43.96
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- in-context learning, dense retrievers, training data selection, pre-trained language models, annotated dataset, meaning representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
