The DOI matches its DataCite record; 25 of the 28 references it cites were found in a registry or an index.
Generalization through Memorization: Nearest Neighbor Language Models
Open accessAbstract
We introduce $k$NN-LMs, which extend a pre-trained neural language model (LM) by linearly interpolating it with a $k$-nearest neighbors ($k$NN) model. The nearest neighbors are computed according to distance in the pre-trained LM embedding space, and can be drawn from any text collection, including the original LM training data. Applying this augmentation to a strong Wikitext-103 LM, with neighbors drawn from the original training set, our $k$NN-LM achieves a new state-of-the-art perplexity of 15.79 - a 2.9 point improvement with no additional training. We also show that this approach has implications for efficiently scaling up to larger training sets and allows for effective domain adaptation, by simply varying the nearest neighbor datastore, again without further training. Qualitatively, the model is particularly helpful in predicting rare patterns, such as factual knowledge. Together, these results strongly suggest that learning similarity between sequences of text is easier than predicting the next word, and that nearest neighbor search is an effective approach for language modeling in the long tail.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“d ( · , · ) is a distance function between the two vectors, and L 2 was shown to be more effective than other alternatives (Khandelwal et al., 2019).”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This model is particularly notable for its large improvements in performance it outperforms the previous best parametric LMs by a large margin in standard language modeling benchmarks, in domain adaptation settings, and on other conditional generation tasks such as machine translation (Khandelwal et al., 2020).”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints.”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“The k NN-LMcomputes the probability of the next token by interpolating a parametric LM with a distribution calculated from the k nearest context-token pairs in the datastore, as demonstrated in Figure 2.”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Note that the dataset to contruct the datastore may not necessarily be the training data that trains the parametric NLM in Eq. 1 - a separate dataset may be used for the datastore construction which would lead to potential applications such as training-free domain adaptation or a gradient-free way to utilize extra training data (Khandelwal et al., 2019).”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“The perplexity continues improving as k grows as shown in (Khandelwal et al., 2019) and confirmed by us.”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Note that the application of k NN-LM requires no additional training; the parameters of the NLM remain as-is, and Eq. 1 is only applied at test time.”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Khandelwal et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Self-citationXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,058
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264
Transformer-XL: Attentive Language Models Beyond a Fixed-Length ContextPreprint
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019) · arXiv (Cornell University) · doi:10.4230/lipics.ecrts.2023.7 · Cited by 1,476
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
Research works printed but not linked: 4
- Yoshua Bengio, R´ ejean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of machine learning research , 3(Feb):1137-1155, 2003.
- Edouard Grave, Moustapha M Cisse, and Armand Joulin. Unbounded cache model for online language modeling with open vocabulary. In NIPS , pp. 6042-6052, 2017a.
- Kelvin Guu, Tatsunori B Hashimoto, Yonatan Oren, and Percy Liang. Generating sentences by editing prototypes. Transactions of the Association for Computational Linguistics , 6:437-450, 2018.
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. In NeurIPS , 2019.
Cited by
Sorted by citations
Items 101–200 of 305
Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis
Zhang, R., Wang, L., Guo, Z., & Shi, J. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00130 · Cited by 17
Chunk-based Nearest Neighbor Machine Translation
Martins, P. H., Marinho, Z., & Martins, A. F. T. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.284 · Cited by 21
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering
Chen, W., Verga, P., de Jong, M., Wieting, J., & Cohen, W. W. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.117 · Cited by 28
Non-parametric Online Learning from Human Feedback for Neural Machine Translation
Wang, D., Wei, H., Zhang, Z., Huang, S., Xie, J., & Chen, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21395 · Cited by 18
DREAM: Improving Situational QA by First Elaborating the Situation
Gu, Y., Dalvi, B. B., & Clark, P. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.82 · Cited by 20
kNN-TL: k-Nearest-Neighbor Transfer Learning for Low-Resource Neural Machine Translation
Liu, S., Liu, X., Wong, D. F., Li, Z., Jiao, W., Chao, L. S., & Zhang, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.105 · Cited by 16
Neural Machine Translation with Monolingual Translation MemoryPreprint
Cai, D., Yan, W., Li, H., Lam, W. P., & Liu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11269 · Cited by 16
Yu, Z., Li, S., Sun, L., Liu, L., & Haining, W. (2022) · Connection Science · doi:10.1080/09540091.2021.2024510 · Cited by 15
∞-former: Infinite Memory Transformer-former: Infinite Memory Transformer
Martins, P. H., Marinho, Z., & Martins, A. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.375 · Cited by 15
Retrosynthesis Prediction with Local Template Retrieval
Xie, S., Yan, R., Guo, J., Xia, Y., Wu, L., & Qin, T. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25664 · Cited by 22
Enhancing Grammatical Error Correction Systems with Explanations
Fei, Y., Cui, L., Yang, S., Lam, W. P., Lan, Z., & Shi, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.413 · Cited by 23
Ryoo, M. S., Gopalakrishnan, K., Kahatapitiya, K., Xiao, T. G., Rao, K., Stone, A., Lu, Y., Ibarz, J., & Arnab, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01828 · Cited by 21
Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer
Jiang, Z., Gao, L., Wang, Z., Araki, J., Ding, H., Callan, J., & Neubig, G. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.149 · Cited by 38
The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Models
Pradeep, R., Thakur, N., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I., Dang, H. T., & Lin, J. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3730090 · Cited by 14
Training Trajectories of Language Models Across Scales
Xia, M., Artetxe, M., Zhou, C., Lin, X. V., Pasunuru, R., Chen, D., Zettlemoyer, L., & Stoyanov, V. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.767 · Cited by 49
Self-citationLMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting
Ramos, R., Martins, B., & Elliott, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.104 · Cited by 22
Retrieving Multimodal Prompts for Generative Visual Question Answering
Ossowski, T., & Hu, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.158 · Cited by 19
Yu, G., Liu, L., Jiang, H., Shi, S., & Ao, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.303 · Cited by 14
RAMP: Retrieval-Augmented MOS Prediction via Confidence-based Dynamic Weighting
Wang, H., Zhao, S., Zheng, X., & Qin, Y. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-851 · Cited by 13
REMED: Retrieval-Augmented Medical Document Query Responding with Embedding Fine-Tuning
Pang, T., Tan, K., Yao, Y., Liu, X., Meng, F., Fan, C., & Zhang, X. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651011 · Cited by 14
Paraphrastic Representations at Scale
Wieting, J., Gimpel, K., Neubig, G., & Berg-kirkpatrick, T. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2022.emnlp-demos.38 · Cited by 13
MetaTroll: Few-shot Detection of State-Sponsored Trolls with Transformer Adapters
Tian, L., Zhang, X., & Lau, J. H. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583417 · Cited by 13
RetroNLU: Retrieval Augmented Task-Oriented Semantic Parsing
Gupta, V., Shrivastava, A., Sagar, A., Aghajanyan, A., & Savenkov, D. (2022) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2022.nlp4convai-1.15 · Cited by 13
Large Search Model: Redefining Search Stack in the Era of LLMs
Wang, L., Yang, N., Huang, X., Yang, L., Majumder, R., & Wei, F. (2023) · ACM SIGIR Forum · doi:10.1145/3642979.3643006 · Cited by 19
Memories are One-to-Many Mapping Alleviators in Talking Face Generation
Tang, A., He, T., Tan, X., Ling, J., Li, R., Zhao, S., Bian, J., & Song, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3409380 · Cited by 12
An Empirical Study of Retrieval Augmented Generation with Chain-of-Thought
Zhao, Y., Cao, H., Zhao, X., & Ou, Z. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800207 · Cited by 16
KNN-CTC: Enhancing ASR via Retrieval of CTC Pseudo Labels
Zhou, J., Zhao, S., Liu, Y., Zeng, W., Chen, Y., & Qin, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447075 · Cited by 11
Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval
Rubin, O., & Berant, J. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00693 · Cited by 38
Guo, C., Tian, Z., Tang, J., Li, S., & Wang, T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103978 · Cited by 15
Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation
Bonetta, G., Cancelliere, R., Liu, D., & Vozila, P. (2021) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.v34i1.128369 · Cited by 15
An Empirical Study of Memorization in NLP
Zheng, X., & Jiang, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.434 · Cited by 12
Exploring Explainable Selection to Control Abstractive Summarization
Wang, H., Gao, Y., Bai, Y., Lapata, M., & Huang, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17641 · Cited by 12
Ben-Zion, D., Gabitov, E., Prior, A., & Bitan, T. (2021) · Neurobiology of Language · doi:10.1162/nol_a_00060 · Cited by 12
Better Language Model with Hypernym Class Prediction
Bai, H., Wang, T., Sordoni, A., & Shi, P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.96 · Cited by 11
To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion
Li, R., Chen, X., Li, C., Shen, Y., Zhao, J., Wang, Y., Han, W., Sun, H., Deng, W., Zhang, Q., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.349 · Cited by 11
You can’t pick your neighbors, or can you? When and How to Rely on Retrieval in the kNN-LM
Drozdov, A., Wang, S., Rahimi, R., McCallum, A., Zamani, H., & Iyyer, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.218 · Cited by 22
Communication-Efficient Personalized Federated Learning for Speech-to-Text Tasks
Du, Y., Zhang, Z., Yue, L., Huang, X., Zhang, Y., Xu, T., Xu, L., & Chen, E. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447662 · Cited by 10
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors
Ivison, H., Smith, N. A., Hajishirzi, H., & Dasigi, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.576 · Cited by 23
Tan, Z., Huang, S., Jia, Z., Cai, J., Li, Y., Lü, W., Zhuang, Y., Tu, K., Xie, P., & Huang, F. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.277 · Cited by 22
Evaluating word representations in a technical language processing pipeline
Nandyala, A. V., Lukens, S., Rathod, S., & Agarwal, P. (2021) · PHM Society European Conference · doi:10.36001/phme.2021.v6i1.2894 · Cited by 12
Automated Fact-Checking of Climate Claims with Large Language ModelsPreprint
Leippold, M., Vaghefi, S. A., Muccione, V., Bingler, J., Stammbach, D., Senni, C. C., Ni, J., Wekhof, T., Yu, T., Schimanski, T., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4731802 · Cited by 10
Prompt-Guided Retrieval Augmentation for Non-Knowledge-Intensive Tasks
Guo, Z., Cheng, S., Wang, Y., Li, P., & Liu, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.693 · Cited by 26
Continual Contrastive Finetuning Improves Low-Resource Relation Extraction
Zhou, W., Zhang, S., Naumann, T., Chen, M., & Poon, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.739 · Cited by 11
SteLLA: A Structured Grading System Using LLMs with RAG
Qiu, H., White, B. F., Ding, A., da Costa, R. F., Hachem, A., Ding, W., & Chen, P. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825385 · Cited by 17
Ice‐kNN‐South: A Lightweight Machine Learning Model for Antarctic Sea Ice Prediction
Lin, Y., Yang, Q., Li, X., Dong, X., Luo, H., Nie, Y., Wang, J., Wang, Y., & Min, C. (2025) · Journal of Geophysical Research Machine Learning and Computation · doi:10.1029/2024jh000433 · Cited by 9
Νικολακόπουλος, Α., Λίτκε, Α., Psychas, A., Veroni, E., & Varvarigou, T. A. (2025) · IEEE Access · doi:10.1109/access.2025.3556973 · Cited by 10
Retrieval-augmented code completion for local projects using large language models
Hostnik, M., & Robnik‐Šikonja, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128596 · Cited by 13
Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition
Wang, X., Zhao, S., & Qin, Y. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-842 · Cited by 9
Revisiting Simple Neural Probabilistic Language Models
Sun, S., & Iyyer, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.407 · Cited by 9
Data-to-text Generation by Splicing Together Nearest Neighbors
Wiseman, S., Bačkurs, A., & Stratos, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.352 · Cited by 12
GNN-SL: Sequence Labeling Based on Nearest Examples via GNN
Wang, S., Meng, Y., Ouyang, R., Li, J., Zhang, T., Lyu, L., & Wang, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.803 · Cited by 9
Li, S., Han, C., Yu, P., Edwards, C., Li, M., Wang, X., Fung, Y. R., Yu, C., Tetreault, J., Hovy, E. H., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.799 · Cited by 11
BehnamGhader, P., Miret, S., & Reddy, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1036 · Cited by 23
Textomics: A Dataset for Genomics Data Summary Generation
Wang, M., Liu, Z., & Wang, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.335 · Cited by 8
Subset Retrieval Nearest Neighbor Machine Translation
Deguchi, H., Watanabe, T., Matsui, Y., Utiyama, M., Tanaka, H., & Sumita, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.10 · Cited by 8
Naseem, U., Thapa, S., Zhang, Q., Hu, L., Masood, A., & Nasim, M. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.32 · Cited by 8
Efficient Nearest Neighbor Emotion Classification with BERT-whitening
Yin, W., & Shang, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.312 · Cited by 8
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_2 · Cited by 12
Liu, D., Yuan, G., Tan, H., Jiang, Y., Bi, H., & Cheng, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3436838 · Cited by 6
Nearest Neighbor Multivariate Time Series Forecasting
Zhang, H., Nie, P., Sun, L., & Boulet, B. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3490603 · Cited by 7
Adaptation Approaches for Nearest Neighbor Language Models
Bhardwaj, R., Polovets, G., & Sunkara, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.73 · Cited by 7
A Canonicalization-Enhanced Known Fact-Aware Framework For Open Knowledge Graph Link Prediction
Wang, Y., Hu, M., Huang, Z., Li, D., Luo, W., Yang, D., & Lu, X. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/259 · Cited by 7
Retrieval Augmentation for Deep Neural Networks
Ramos, R., Pereira, P., Moniz, H., Carvalho, J. P., & Martins, B. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533978 · Cited by 7
XDailyDialog: A Multilingual Parallel Dialogue Corpus
Liu, Z., Nie, P., Cai, J., Wang, H., Niu, Z.-Y., Zhang, P., Sachan, M., & Peng, K. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.684 · Cited by 9
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Wang, Z., Pan, X., Yu, D., Yu, D., Chen, J., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.246 · Cited by 13
Generating Video Game Scripts with Style
Latouche, G. L., Marcotte, L., & Swanson, B. (2023) · Workshop on NLP for Conversational AI (NLP4ConvAI) · doi:10.18653/v1/2023.nlp4convai-1.11 · Cited by 7
Emptying the Ocean with a Spoon: Should We Edit Models?
Pinter, Y., & Elhadad, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1012 · Cited by 17
High Performance Natural Language Processing
Ilharco, G., Magalhaes, C. I., Turc, I., Dettmers, T., Ferreira, F., & Lee, K. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2020.emnlp-tutorials.4 · Cited by 7
Improving edit-based unsupervised sentence simplification using fine-tuned BERT
Rashid, M. A., & Amirkhani, H. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.01.009 · Cited by 7
EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation
Keh, S. S., Bharadwaj, R., Liu, E., Tedeschi, S., Gangal, V., & Roberto Navigli, A. (2022) · Workshop on Figurative Language Processing (FigLang) · doi:10.18653/v1/2022.flp-1.15 · Cited by 7
ProtEx: A Retrieval-Augmented Approach for Protein Function PredictionPreprint
Shaw, P., Gurram, B., Belanger, D., Gane, A., Bileschi, M. L., Colwell, L. J., Toutanova, K., & Parikh, A. P. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.30.596539 · Cited by 8
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
Kuratov, Y., Bulatov, A., Anokhin, P., Rodkin, I., Sorokin, D., Sorokin, A., & Burtsev, M. (2024) · arXiv · doi:10.48550/arxiv.2406.10149 · Cited by 19
“Some works focus on directly retrieving predictions (Khandelwal et al., 2019).”
BackgroundTrusting Your Evidence: Hallucinate Less with Context-aware Decoding
Shi, W., Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L., & Yih, S. W.-T. (2023) · arXiv · doi:10.48550/arxiv.2305.14739 · Cited by 6
“MMI-based decoding (Li et al., 2015) uses a contrastive formulation to improve output diversity in dialog generation.”
BackgroundSelf-citationA Deep Learning Approach for Automated Image Captioning with CNN and LSTM Models
C, S. S. K., Natarajan, S. K., & Ambika, I. (2025) · International Conference on Computational Robotics, Testing and Engineering Evaluation (ICCRTEE) · doi:10.1109/iccrtee64519.2025.11053022 · Cited by 6
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models
Pozzobon, L., Ermiş, B., Lewis, P., & Hooker, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.339 · Cited by 19
Subset Retrieval Nearest Neighbor Machine Translation
Deguchi, H., Watanabe, T., Matsui, Y., Utiyama, M., Tanaka, H., & Sumita, E. (2024) · Journal of Natural Language Processing · doi:10.5715/jnlp.31.374 · Cited by 6
Textual Differential Privacy for Context-Aware Reasoning with Large Language Model
Yu, J., Zhou, J., Ding, Y., Zhang, L., Guo, Y., & Satō, H. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00135 · Cited by 7
M2R-Whisper: Multi-stage and Multi-scale Retrieval Augmentation for Enhancing Whisper
Zhou, J., Zhao, S., He, J., Wang, H., Zeng, W., Chen, Y., Sun, H., Kong, A., & Qin, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888086 · Cited by 6
Long-term Control for Dialogue Generation: Methods and Evaluation
Ramakrishnan, R., Narangodage, H., Schilman, M., Weinberger, K. Q., & McDonald, R. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.54 · Cited by 8
Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction
Wan, Z., Liu, Q., Mao, Z., Cheng, F., Kurohashi, S., & Li, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.113 · Cited by 6
Code Vulnerability Detection via Nearest Neighbor Mechanism
Du, Q., Kuang, X., & Zhao, G. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.459 · Cited by 6
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories
Ge, S., Xiong, C., Rosset, C., Overwijk, A., Han, J., & Bennett, P. N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.111 · Cited by 10
Retrieval augmentation for text-to-table generation
Zhang, J., Tan, M., Zhang, J., Zhang, X., Zhou, J., & Li, C. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104135 · Cited by 5
MIL-Decoding: Detoxifying Language Models at Token-Level via Multiple Instance Learning
Zhang, X., & Wan, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.11 · Cited by 6
On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 40
Improving Zero-Shot Chinese-English Code-Switching ASR with kNN-CTC and Gated Monolingual Datastores
Zhou, J., Zhao, S., Wang, H., Zhang, T., Sun, H., Wang, X., & Qin, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890024 · Cited by 5
CLAP-S: Support Set Based Adaptation for Downstream Fiber-optic Acoustic Recognition
Sun, J., Han, S., Kohno, W., & Chen, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887829 · Cited by 5
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification
Bari, M. S., Haider, B. A., & Mansour, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.131 · Cited by 6
Improving Aspect Sentiment Classification via Retrieving from Training Data
Ling, T., Chen, L., Liao, C., Huang, S., Yu, Z., & Liu, Y. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317451 · Cited by 5
kNN-LM Does Not Improve Open-ended Text Generation
Wang, S., Song, Y., Drozdov, A., Garimella, A., Manjunatha, V., & Iyyer, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.929 · Cited by 12
STAR-TM: STructure Aware Reconstruction of Textured Mesh From Single Image
Wu, T., Gao, L., Zhang, L., Lai, Y., & Zhang, H. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305630 · Cited by 5
Retrieved Sequence Augmentation for Protein Representation LearningPreprint
Ma, C., Zhao, H., Zheng, L., Xin, J., Li, Q., Wu, L., Deng, Z., Yang, L., Liu, Q., & Kong, L. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.22.529597 · Cited by 7
Amgpt: A Large Language Model for Contextual Querying in Additive ManufacturingPreprint
Chandrasekhar, A., Chan, J., Ogoke, F., Ajenifujah, O., & Farimani, A. B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4870552 · Cited by 5
Yakdan, S. M., Warner, B., Ghogawala, Z., Ray, W. Z., Bydon, M., Steinmetz, M. P., Griffey, R. T., Foraker, R., Wilcox, A., Lu, C., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02337-7 · Cited by 4
Structured Case-Based Reasoning for Inference-Time Adaptation of Text-to-SQL Parsers
Awasthi, A., Chakrabarti, S., & Sarawagi, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26476 · Cited by 4
Retrieval-Augmented Domain Adaptation of Language Models
Xu, B., Zhao, C., Jiang, W., Zhu, P., Dai, S., Pang, C., Sun, Z., Wang, S., & Sun, Y. (2023) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2023.repl4nlp-1.5 · Cited by 10
Nearest Neighbor Language Models for Stylistic Controllable Generation
Trotta, S., Flek, L., & WELCH, C. A. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.25 · Cited by 6
Relation Inference Enhancement Network for Visual Commonsense Reasoning
Yuan, M., Jia, G., & Bao, B.-K. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521725 · Cited by 4
Ascent Similarity Caching With Approximate Indexes
Salem, T. S., Neglia, G., & Damiano Carra, B. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3217012 · Cited by 4
Metadata
- Authors
- Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., & Lewis, M.
- Affiliations
- Meta (United States) (US)
- Meta (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1911.00172
- OpenAlex
- W2988841832
- Cited by
- 285
- References
- 28
- Citations by year
- 2026: 1 · 2025: 34 · 2024: 34 · 2023: 111 · 2022: 52 · 2021: 39 · 2020: 13 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Styrelsen för Internationellt Utvecklingssamarbete
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- linear interpolation, pre-trained language models, embedding space, perplexity reduction, WikiText-103, domain adaptation, factual knowledge, next-word prediction, sequence similarity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
