The DOI matches its Crossref record; the references it cites are not checked yet.
REPLUG: Retrieval-Augmented Black-Box Language Models
Open accessAbstract
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, most RAG systems still retrieve unstructured chunks and rely on one-shot generation, which often yields redundant context, low information density, and brittle multi-hop reasoning.”
From TaSR-RAG: Taxonomy-guided Structured Reasoning for Retrieval-Augmented Generation (2026) · cites this paper as [19]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This reduces the computational burden of finetuning large models (e.g., 405B parameters) by minimizing the KL divergence between retrieval likelihood and the model's output perplexity.”
From The AI Hippocampus: How Far are We From Human Memory? (2026) · cites this paper as Shi et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 31
- Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Si- monyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Bal- timore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 2206-2240.
- PMLR.
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sas- try, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cum- mings, Matthias Plappert, Fotios Chantzis, Eliza- beth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluat- ing large language models trained on code. ArXiv preprint, abs/2107.03374. · arXiv
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. ArXiv preprint, abs/2204.02311. · arXiv
- Hyung Won Chung, Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022a. Scaling instruction-finetuned language mod- els. ArXiv preprint, abs/2210.11416. · arXiv
- Hyung Won Chung, Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022b. Scaling instruction-finetuned language mod- els. ArXiv preprint, abs/2210.11416. · arXiv
- Leo Gao, Stella Biderman, Sid Black, Laurence Gold- ing, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2021. The Pile: An 800gb dataset of diverse text for language modeling. ArXiv preprint, abs/2101.00027. · arXiv
- Anirudh Goyal, Abram L. Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adrià Puig- domènech Badia, Arthur Guez, Mehdi Mirza, Peter C. Humphreys, Ksenia Konyushkova, Michal Valko, Si- mon Osindero, Timothy P. Lillicrap, Nicolas Heess, and Charles Blundell. 2022. Retrieval-augmented re- inforcement learning. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Bal- timore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 7740-7765.
- PMLR.
- Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Retrieval augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 3929-3938. PMLR.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Men- sch, Elena Buchatskaya, Trevor Cai, Eliza Ruther- ford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Train- ing compute-optimal large language models. ArXiv preprint, abs/2203.15556. · arXiv
- Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, and Jiebo Luo. 2022. Promptcap: Prompt-guided task-aware image captioning. ArXiv preprint, abs/2211.09699. · arXiv
- Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebas- tian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022a. Unsupervised dense informa- tion retrieval with contrastive learning. Transactions on Machine Learning Research.
- Gautier Izacard, Patrick Lewis, Maria Lomeli, Lu- cas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022b. Few-shot learning with re- trieval augmented language models. ArXiv preprint, abs/2208.03299. · arXiv
- Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535-547.
- Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
- Patrick S. H. Lewis, Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neu- ral Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
- Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2022. When not to trust language models: Inves- tigating effectiveness and limitations of paramet- ric and non-parametric memories. ArXiv preprint, abs/2212.10511. · arXiv
- Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture mod- els. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open- Review.net.
- Sewon Min, Weijia Shi, Mike Lewis, Xilun Chen, Wen- tau Yih, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Nonparametric masked language modeling. ArXiv preprint, abs/2212.01349. · arXiv
- Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022. Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9844-9855, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
- Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and be- yond. Foundations and Trends® in Information Re- trieval, 3(4):333-389.
- Teven Le Scao, Angela Fan, Christopher Akiki, El- lie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b- parameter open-access multilingual language model. ArXiv preprint, abs/2211.05100. · arXiv
- Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, and Li- juan Wang. 2023. Prompting GPT-3 to be reliable. In Proc. of ICLR.
- Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Richard James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-Tau Yih. 2023. Retrieval-augmented multimodal language modeling. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 39755-39769.
- PMLR.
- Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. ArXiv preprint, abs/2205.01068. · arXiv
- Zexuan Zhong, Tao Lei, and Danqi Chen. 2022. Train- ing language models with memory augmentation. In Proceedings of the 2022 Conference on Empir- ical Methods in Natural Language Processing, pages 5657-5673, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Cited by
Sorted by citations
Items 201–219 of 219
An LLM-Powered Intelligent Assistant for Learning Management Systems Using RAG and NL2SQL Pipelines
Acharya, S. S., Shenoy, G. S., Shenai, S., GudurNitte, S., & Tushar, D. S. (2026) · International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE) · doi:10.1109/icdcece68961.2026.11646016
Pandilova, E., Petrov, M., Chorbev, I., Gjorgjevikj, D., & Trajanov, D. (2026) · Information · doi:10.3390/info17080790
Kim, H., Moon, J., & Baik, N. (2026) · Electronics · doi:10.3390/electronics15163643 · Cited by 1
Sheng, Z., Sun, R., Xu, J., Luo, H., Chen, P., Zhang, W., & Cui, B. (2026) · Proceedings of the VLDB Endowment · doi:10.14778/3819518.3819546
CGR: A Budget-Aware Closed-Loop Framework for Efficient Knowledge Graph Question Answering
Li, Y., Zhang, H., Wang, J., Chen, J., & Xu, L. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-4805-6_14
OnkoRAG: Modelling Cross-Lingual German-Arabic Medical Question Answering
Jennings, N. G., Petukhova, V. V., & Klakow, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37249-9_54
SPOT: Map-grounded camera handoff for unsupervised multi-camera vehicle tracking
roh, Y., Park, I. J., & Hwang, C. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114954
Liu, H., Zhao, R., Zhao, N., Qian, L., & Zhou, J. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3739198
Barad, K., Gunjan, R., & Butalia, A. (2026) · International Conference on Innovation & Growth in Next-Gen Intelligent Technology & Engineering (IGNITE) · doi:10.1109/ignite69592.2026.11705780
Li, Z., Yao, L., Zhu, Z., Zeng, J., & Song, Z. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3739955
S, R. M., Santosh, D. T., Joshi, J., Kumar, B., Naveen, N., & Satyamurty, C. V. S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-6513-8_33
Fossilized Memories: Adapting Open LLMs to Oil and Gas Data While Mitigating Catastrophic Forgetting
Raza, S. A., Alwalmany, M. M., Abusnina, A. M., & Alutaibi, K. A. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-28469-3_17
Zhu, J., Wu, H., & Cao, Y. (2026) · Electronics · doi:10.3390/electronics15204759
Adaptive Fault-Tolerant Retrieval-Augmented Generation for Multi-Hop Open-Domain Question Answering
Liu, P., Li, B., Li, S., Ge, X., Ran, Z., & Li, K. (2026) · International Conference of Data and Information Systems (DIS) · doi:10.1109/dis72268.2026.11725720
A Conceptual Framework for Characterising and Analysing Retrieval–Augmented Generation Techniques
Igbekele Emmanuel, O., & Daramola, O. J. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-42763-2_20
Kurumsal Bilgi Yönetiminde Büyük Dil Modelleri ve Retrieval-Augmented Generation Yaklaşımı
Bayraktar, C. (2026) · Özgür Yayınları eBooks · doi:10.58830/ozgur.pub1539.c6222
Dynamic Retrieval-Reasoning Framework on Hypergraphs for Complex Question Answering
Hou, Y., & Zheng, C. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728690
MCIR-RAG: Multi-Agent Chain-of-Thought Iterative Reflection for Multi-modal Document Understanding
Liu, C., Wang, S., Li, Z., Hu, M., Fang, Q., Xiao, Y., Zhang, J., Luo, Z., Luo, W., Yi, Z., et al. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729376
Context Selection as a Partial Order: A Blackwell Framework and Verified LLM Evidence
Bilous, A., Lytvyn, V., Pukach, P. Y., & Rybchak, Z. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8100336
Metadata
- Authors
- Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., & Yih, W.-T.
- Affiliations
- University of Washington (US)
- Stanford University (US)
- Korea Advanced Institute of Science and Technology (KR)
- Venue
- Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 8371-8384
- OpenAlex
- W4401042773
- Cited by
- 209
- References
- 0
- Citations by year
- 2026: 79 · 2025: 96 · 2024: 32
- FWCI
- 59.77
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- retrieval-augmented generation, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
