CoChat
Sign Up
Cited by 48Topic ModelingInformation Retrieval and Search BehaviorAdvanced Image and Video Retrieval Techniques

Accelerating Retrieval-Augmented Generation

Open access
Quinn, Derrick, Nouri, Mohammad, Patel, Neel, Salihu, John, Salemi, Alireza, Lee, Sukhan, Zamani, Hamed, Alian, Mohammad·2025·Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1
DOI 10.1145/3669940.3707264
Open this paper in CoChat — free accountOpen PDF

Abstract

An evolving solution to address hallucination and enhance accuracy in large language models (LLMs) is Retrieval-Augmented Generation (RAG), which involves augmenting LLMs with information retrieved from an external knowledge source, such as the web. This paper profiles several RAG execution pipelines and demystifies the complex interplay between their retrieval and generation phases. We demonstrate that while exact retrieval schemes are expensive, they can reduce inference time compared to approximate retrieval variants because an exact retrieval model can send a smaller but more accurate list of documents to the generative model while maintaining the same end-to-end accuracy. This observation motivates the acceleration of the exact nearest neighbor search for RAG.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3669940.3707264

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Vector-based storage remains the dominant paradigm in non-parametric agent memory, enabling efficient approximate nearest-neighbor search over episodic or semantic memory in retrieval-augmented agents.

    “Vector-based storage remains the dominant paradigm in non-parametric agent memory, enabling efficient approximate nearest-neighbor search over episodic or semantic memory in retrievalaugmented agents (Guu et al., 2020; Quinn et al., 2025).”

    From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Quinn et al., 2025

    Section 2.1 says RAG documents are stored as embedding vectors in a vector database and identifies approximate nearest-neighbor search as an efficiency-oriented retrieval method. Section 3.2 likewise describes approximate search over those vectors. The paper does not establish that vector storage is dominant in non-parametric agent memory or discuss episodic and semantic agent memory. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Vector index is the dominant implementation of external memory in current work of agents or LLMs, utilized within the RAG framework.

    “The dominant implementation of external memory in the current work of agents or LLMs is the vector database, utilized within the RAG framework (Quinn et al., 2025; Li et al., 2025i; Zhang et al., 2025l).”

    From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Quinn et al., 2025

    The Introduction describes vector databases as stores for document embeddings in current RAG retrieval approaches, and §2.1 says real RAG implementations store embeddings in a vector database. The paper does not establish that vector databases are the dominant implementation of external memory across current agent or LLM work. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 97

  1. Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwa- tra, Bhargav S. Gulavani, and Ramachandran Ramjee. 2023. SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills. Microsoft Research Blog (September 2023). https://doi.org/10.48550/arXiv.2308.16369 · DOI · Paper page
  2. Yeonchan Ahn, Sang-Goo Lee, Junho Shim, and Jaehui Park. 2022. Retrieval-Augmented Response Generation for Knowledge-Grounded Conversation in the Wild. IEEE Access 10 (2022), 131374-131385. https://doi.org/10.1109/ACCESS.2022.3228964 · DOI
  3. Mohammad Alian, Seung Won Min, Hadi Asgharimoghaddam, Ashutosh Dhar, Dong Kai Wang, Thomas Roewer, Adam McPad- den, Oliver O'Halloran, Deming Chen, Jinjun Xiong, et al. 2018. Application-Transparent Near-Memory Processing Architecture with Memory Channel Network. In 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 802-814. https://doi.org/10.1109/MICRO.2018.00070 · DOI
  4. Martin Aumüller, Erik Bernhardsson, and Alexander Faithfull. 2020. ANN-Benchmarks: A benchmarking tool for approximate nearest neighbor algorithms. Information Systems 87 (2020), 101374. https://doi.org/10.48550/arXiv.1807.05614 · DOI
  5. Artem Babenko and Victor Lempitsky. 2012. The inverted multi-index. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. 3069-3076. https://doi.org/10.1109/CVPR.2012.6248038 · DOI
  6. Giovanni Bonetta, Rossella Cancelliere, Ding Liu, and Paul Vozila. 2021. Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation. The International FLAIRS Conference Proceedings 34, 1 (April 2021). https://doi.org/10.32473/flairs.v34i1.128369 · DOI
  7. Francesco Busolin, Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, and Salvatore Trani. 2024. Early Exit Strategies for Approximate k-NN Search in Dense Retrieval. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (Boise, ID, USA) (CIKM '24). Associ- ation for Computing Machinery, New York, NY, USA, 3647-3652. https://doi.org/10.1145/3627673.3679903 · DOI
  8. Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. 2019. Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 1219-1228. https://doi.org/10.18653/v1/N19-1124 · DOI
  9. Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William Cohen. 2022. MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 5558-5570. https://doi.org/10.18653/v1/2022.emnlp-main.375 · DOI
  10. Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W. Cohen. 2022. Re-Imagen: Retrieval-Augmented Text-to- Image Generator. ArXiv abs/2209.14491 (2022). https: //api.semanticscholar.org/CorpusID:252596087 · arXiv
  11. William J. Dally, Yatish Turakhia, and Song Han. 2020. Domain- specific hardware accelerators. Commun. ACM 63, 7 (2020), 48-57. https://doi.org/10.1145/3361682 · DOI
  12. Feyza Duman Keles, Pruthuvi Mahesakya Wijewardena, and Chinmay Hegde. 2023. On The Computational Complexity of Self-Attention. In Proceedings of The 34th International Conference on Algorithmic Learning Theory (Proceedings of Machine Learning Research, Vol. 201), Shipra Agrawal and Francesco Orabona (Eds.). PMLR, 597-619. https://proceedings.mlr.press/v201/duman-keles23a.html
  13. Zhengcong Fei. 2021. Memory-Augmented Image Captioning. Proceedings of the AAAI Conference on Artificial Intelligence 35, 2 (May 2021), 1317-1324. https://doi.org/10.1609/aaai.v35i2.16220 · DOI
  14. Amin Firoozshahian, Joel Coburn, Roman Levenstein, Rakesh Nattoji, Ashwin Kamath, Olivia Wu, Gurdeepak Grewal, Harish Aepala, Bhasker Jakka, Bob Dreyer, Adam Hutchin, Utku Diril, Krishnakumar Nair, Ehsan K. Aredestani, Martin Schatz, Yuchen Hao, Rakesh Komu- ravelli, Kunming Ho, Sameer Abu Asal, Joe Shajrawi, Kevin Quinn, Nagesh Sreedhara, Pankaj Kansal, Willie Wei, Dheepak Jayaraman, Linda Cheng, Pritam Chopda, Eric Wang, Ajay Bikumandla, Arun Karthik Sengottuvel, Krishna Thottempudi, Ashwin Narasimha, Brian Dodds, Cao Gao, Jiyuan Zhang, Mohammed Al-Sanabani, Ana Zehtabioskuie, Jordan Fix, Hangchen Yu, Richard Li, Kaustubh Gondkar, Jack Montgomery, Mike Tsai, Saritha Dwarakapuram, Sanjay Desai, Nili Avidan, Poorvaja Ramani, Karthik Narayanan, Ajit Mathews, Sethu Gopal, Maxim Naumov, Vijay Rao, Krishna Noru, Harikrishna Reddy, Prahlad Venkatapuram, and Alexis Bjorlin. 2023. MTIA: First Generation Silicon Targeting Meta's Recommendation Systems. In Proceedings of the 50th Annual International Symposium on Computer Architecture (Orlando, FL, USA) (ISCA '23). Association for Computing Machinery, New York, NY, USA, Article 80, 13 pages. https://doi.org/10.1145/3579371.3589348 · DOI
  15. Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor OK Li. 2018. Search engine guided neural machine translation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, Vol. 32. https://doi.org/10.1609/aaai.v32i1.12013 · DOI
  16. Liangke Gui, Borui Wang, Qiuyuan Huang, Alexander Hauptmann, Yonatan Bisk, and Jianfeng Gao. 2022. KAT: A Knowledge Augmented Transformer for Vision-and-Language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (Eds.). Association for Computational Linguistics, Seattle, United States, 956-968. https://doi.org/10.18653/v1/2022.naacl-main.70 · DOI · Paper page
  17. Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, and Percy Liang. 2018. A Retrieve-and-Edit Framework for Predicting Struc- tured Outputs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montréal, Canada) (NIPS'18).
  18. Curran Associates Inc., Red Hook, NY, USA, 10073-10083. https://dl.acm.org/doi/10.5555/3327546.3327670 · DOI
  19. Qiuxiang He, Guoping Huang, Qu Cui, Li Li, and Lemao Liu. 2021. Fast and Accurate Neural Machine Translation with Translation Memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, Online, 3170-3180. https://doi.org/10.18653/v1/2021.acl-long.246 · DOI
  20. Sebastian Hofstätter, Jiecao Chen, Karthik Raman, and Hamed Zamani. 2023. FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation. In Proceedings of the 46th International ACM SIGIR Con- ference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR '23). Association for Computing Machinery, New York, NY, USA, 1437-1447. https://doi.org/10.1145/3539618.3591687 · DOI · Paper page
  21. Mark Horowitz. 2014. 1.1 Computing's energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC). 10-14. https://doi.org/10.1109/ISSCC.2014.6757323 · DOI
  22. Mohamed Assem Ibrahim, Onur Kayiran, Yasuko Eckert, Gabriel H Loh, and Adwait Jog. 2021. Analyzing and leveraging decoupled L1 caches in GPUs. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 467-478. https://doi.org/10.1109/HPCA51647.2021.00047 · DOI
  23. Gautier Izacard and Edouard Grave. 2021. Distilling Knowl- edge from Reader to Retriever for Question Answering. In International Conference on Learning Representations. https://openreview.net/forum?id=NTEz-6wysdb
  24. Gautier Izacard and Edouard Grave. 2021. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, Online, 874-880. https://doi.org/10.18653/v1/2021.eacl-main.74 · DOI · Paper page
  25. Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022. Deduplicat- ing training data mitigates privacy risks in language models. In International Conference on Machine Learning. PMLR, 10697-10707. https://proceedings.mlr.press/v162/kandpal22a.html
  26. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 6769-6781. https://doi.org/10.18653/v1/2020.emnlp-main.550 · DOI · Paper page
  27. Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. 2024. REALTIME QA: what's the answer right now?. In Proceedings of the 37th International Conference on Neural Information Processing Systems (New Orleans, LA, USA) (NIPS '23).
  28. Curran Associates Inc., Red Hook, NY, USA, Article 2130, 19 pages. https://dl.acm.org/doi/10.5555/3666122.3668252 · DOI
  29. Amirhossein Kazemnejad, Mohammadreza Salehi, and Mahdieh Soleymani Baghshah. 2020. Paraphrase Generation by Learning How to Edit from Samples. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Online, 6010-6021. https://doi.org/10.18653/v1/2020.acl-main.535 · DOI
  30. Liu Ke, Xuan Zhang, Jinin So, Jong-Geon Lee, Shin-Haeng Kang, Sukhan Lee, Songyi Han, YeonGon Cho, Jin Hyun Kim, Yongsuk Kwon, KyungSoo Kim, Jin Jung, Ilkwon Yun, Sung Joo Park, Hyunsun Park, Joonho Song, Jeonghyeon Cho, Kyomin Sohn, Nam Sung Kim, and Hsien-Hsin S. Lee. 2022. Near-Memory Processing in Action: Accelerating Personalized Recommendation With AxDIMM. IEEE Micro 42, 1 (2022), 116-127. https://doi.org/10.1109/MM.2021.3097700 · DOI
  31. Ben Keller, Rangharajan Venkatesan, Steve Dai, Stephen G Tell, Brian Zimmer, Charbel Sakr, William J Dally, C Thomas Gray, and Brucek Khailany. 2023. A 95.6-TOPS/W deep learning inference accelerator with per-vector scaled 4-bit quantization in 5 nm. IEEE Journal of Solid-State Circuits 58, 4 (2023), 1129-1141. https://doi.org/https: //doi.org/10.1109/VLSITechnologyandCir46769.2022.9830277 · DOI
  32. Jin Hyun Kim, Shin-Haeng Kang, Sukhan Lee, Hyeonsu Kim, Yuhwan Ro, Seungwon Lee, David Wang, Jihyun Choi, Jinin So, YeonGon Cho, JoonHo Song, Jeonghyeon Cho, Kyomin Sohn, and Nam Sung Kim. 2022. Aquabolt-XL HBM2-PIM, LPDDR5-PIM With In-Memory Processing, and AXDIMM With Acceleration Buffer. IEEE Micro 42, 3 (2022), 20-30. https://doi.org/10.1109/MM.2022.3164651 · DOI
  33. To Eun Kim, Alireza Salemi, Andrew Drozdov, Fernando Diaz, and Hamed Zamani. 2024. Retrieval-Enhanced Ma- chine Learning: Synthesis and Opportunities. arXiv (2024). https://doi.org/10.48550/arXiv.2407.12982 arXiv:2407.12982. · DOI
  34. Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, and Hamed Zamani. 2024. LongLaMP: A Benchmark for Personalized Long-form Text Generation. arXiv (2024). https://doi.org/10.48550/arXiv.2407.11016 arXiv:2407.11016. · DOI
  35. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain ques- tion answering. arXiv preprint arXiv:1906.00300 (2019). https://doi.org/10.48550/arXiv.1906.00300 · DOI
  36. Yejin Lee, Hyunji Choi, Sunhong Min, Hyunseung Lee, Sangwon Beak, Dawoon Jeong, Jae W. Lee, and Tae Jun Ham. 2022. ANNA: Specialized Architecture for Approximate Nearest Neighbor Search. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). 169-183. https://doi.org/10.1109/HPCA53966.2022.00021 · DOI
  37. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS'20).
  38. Curran Associates Inc., Red Hook, NY, USA, Article 793, 16 pages. https://doi.org/10.5555/3495724.3496517 · DOI
  39. Conglong Li, Minjia Zhang, David G. Andersen, and Yuxiong He. 2020. Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD '20). Association for Computing Machinery, New York, NY, USA, 2539-2554. https://doi.org/10.1145/3318464.3380600 · DOI
  40. Huaicheng Li, Daniel S. Berger, Lisa Hsu, Daniel Ernst, Pantea Zardoshti, Stanko Novakovic, Monish Shah, Samir Rajadnya, Scott Lee, Ishwar Agarwal, Mark D. Hill, Marcus Fontoura, and Ricardo Bianchini. 2023. Pond: CXL-Based Memory Pooling Systems for Cloud Platforms. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (Vancouver, BC, Canada) (ASPLOS 2023). Association for Computing Machinery, New York, NY, USA, 574-587. https://doi.org/10.1145/3575693.3578835 · DOI
  41. Huayang Li, Yixuan Su, Deng Cai, Yan Wang, and Lemao Liu. 2022. A Survey on Retrieval-Augmented Text Generation. ArXiv abs/2202.01110 (2022). https://api.semanticscholar.org/CorpusID: 246472929 · arXiv
  42. Wen Li, Ying Zhang, Yifang Sun, Wei Wang, Mingjie Li, Wenjie Zhang, and Xuemin Lin. 2020. Approximate Nearest Neighbor Search on High Dimensional Data -Experiments, Analyses, and Improvement. IEEE Transactions on Knowledge and Data Engineering 32, 8 (2020), 1475-1488. https://doi.org/10.1109/TKDE.2019.2909204 · DOI
  43. Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Associa- tion for Computational Linguistics, Barcelona, Spain, 74-81. https://aclanthology.org/W04-1013
  44. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. 2023. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv preprint arXiv:2306.00978 (2023). https://doi.org/10.48550/arXiv.2306.00978 · DOI · Paper page
  45. Shangqing Liu, Yu Chen, Xiaofei Xie, Jing Kai Siow, and Yang Liu. 2021. Retrieval-Augmented Generation for Code Summarization via Hybrid GNN. In International Conference on Learning Representations. https://openreview.net/forum?id=zv-typ1gPxA
  46. Gabriel H. Loh, Natalie Enright Jerger, Ajaykumar Kannan, and Yasuko Eckert. 2015. Interconnect-Memory Challenges for Multi-chip, Silicon Interposer Systems. In Proceedings of the 2015 International Symposium on Memory Systems (Washington DC, DC, USA) (MEMSYS '15). Association for Computing Machinery, New York, NY, USA, 3-10. https://doi.org/10.1145/2818950.2818951 · DOI
  47. Yu A. Malkov and D. A. Yashunin. 2020. Efficient and Ro- bust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 4 (apr 2020), 824-836. https://doi.org/10.1109/TPAMI.2018.2889473 · DOI · Paper page
  48. 2 AMD Chiplet Architecture for High-Performance Server and Desktop Products. In 2020 IEEE International Solid-State Circuits Conference -(ISSCC). 44-45. https://doi.org/10.1109/ISSCC19947.2020.9063103 · DOI
  49. Sang-Soo Park, KyungSoo Kim, Jinin So, Jin Jung, Jonggeon Lee, Kyoungwan Woo, Nayeon Kim, Younghyun Lee, Hyungyo Kim, Yongsuk Kwon, Jinhyun Kim, Jieun Lee, YeonGon Cho, Yongmin Tai, Jeonghyeon Cho, Hoyoung Song, Jung Ho Ahn, and Nam Sung Kim. 2024. An LPDDR-based CXL-PNM Platform for TCO-efficient Inference of Transformer-based Large Language Models. In 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). 970-982. https://doi.org/10.1109/HPCA57654.2024.00078 · DOI
  50. Md Rizwan Parvez, Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Retrieval Augmented Code Generation and Summarization. In Findings of the Association for Computational Linguistics: EMNLP 2021, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Compu- tational Linguistics, Punta Cana, Dominican Republic, 2719-2734. https://doi.org/10.18653/v1/2021.findings-emnlp.232 · DOI · Paper page
  51. N. Patel, A. Mamandipoor, M. Nouri, and M. Alian. 2024. SmartDIMM: In-Memory Acceleration of Upper Layer Protocols. In 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE Computer Society, Los Alamitos, CA, USA, 312-329. https://doi.org/10.1109/HPCA57654.2024.00032 · DOI
  52. Neel Patel, Amin Mamandipoor, Derrick Quinn, and Mohammad Alian. 2023. XFM: Accelerated Software-Defined Far Memory. In Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture (Toronto, ON, Canada) (MICRO '23). Asso- ciation for Computing Machinery, New York, NY, USA, 769-783. https://doi.org/10.1145/3613424.3623776 · DOI
  53. Hao Peng, Ankur Parikh, Manaal Faruqui, Bhuwan Dhingra, and Dipanjan Das. 2019. Text Generation with Exemplar-based Adaptive Decoding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 2555-2565. https://doi.org/10.18653/v1/N19-1263 · DOI
  54. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a Benchmark for Knowledge Intensive Language Tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, Online, 2523-2544. https://doi.org/10.18653/v1/2021.naacl-main.200 · DOI · Paper page
  55. Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021. RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, Online, 5835-5847. https://doi.org/10.18653/v1/2021.naacl-main.466 · DOI
  56. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. The Journal of Machine Learning Research 21, 1, Article 140 (jan 2020), 67 pages. https://doi.org/10.5555/3455716.3455856 · DOI
  57. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Jian Su, Kevin Duh, and Xavier Carreras (Eds.). Association for Computational Linguistics, Austin, Texas, 2383-2392. https://doi.org/10.18653/v1/D16-1264 · DOI · Paper page
  58. Rita Ramos, Desmond Elliott, and Bruno Martins. 2023. Retrieval- augmented Image Captioning. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, Andreas Vlachos and Isabelle Augenstein (Eds.). Association for Computational Linguistics, Dubrovnik, Croatia, 3666-3681. https://doi.org/10.18653/v1/2023.eacl-main.266 · DOI
  59. Alireza Salemi, Juan Altmayer Pizzorno, and Hamed Zamani. 2023. A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answering. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR '23). Association for Computing Machinery, New York, NY, USA, 110-120. https://doi.org/10.1145/3539618.3591629 · DOI
  60. Alireza Salemi, Surya Kallumadi, and Hamed Zamani. 2024. Op- timization Methods for Personalizing Large Language Models through Retrieval Augmentation. In Proceedings of the 47th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR '24). Associ- ation for Computing Machinery, New York, NY, USA, 752-762. https://doi.org/10.1145/3626772.3657783 · DOI · Paper page
  61. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024. LaMP: When Large Language Models Meet Per- sonalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),
  62. Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 7370-7392. https://doi.org/10.18653/v1/2024.acl-long.399 · DOI · Paper page
  63. Alireza Salemi, Mahta Rafiee, and Hamed Zamani. 2023. Pre-Training Multi-Modal Dense Retrievers for Outside-Knowledge Visual Ques- tion Answering. In Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval (Taipei, Taiwan) (ICTIR '23). Association for Computing Machinery, New York, NY, USA, 169-176. https://doi.org/10.1145/3578337.3605137 · DOI
  64. Alireza Salemi and Hamed Zamani. 2024. Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models. arXiv:2409.09510 [cs.CL] https://arxiv.org/abs/2409.09510 · arXiv
  65. Alireza Salemi and Hamed Zamani. 2024. Evaluating Retrieval Quality in Retrieval-Augmented Generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR '24). Association for Computing Machinery, New York, NY, USA, 2395-2400. https://doi.org/10.1145/3626772.3657957 · DOI
  66. Alireza Salemi and Hamed Zamani. 2024. Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Max- imization. arXiv:2410.09942 [cs.CL] https://arxiv.org/abs/2410.09942 · arXiv
  67. Alireza Salemi and Hamed Zamani. 2024. Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval- Augmented Large Language Models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR '24). Asso- ciation for Computing Machinery, New York, NY, USA, 741-751. https://doi.org/10.1145/3626772.3657733 · DOI
  68. Satyabrata Sarangi and Bevan Baas. 2021. DeepScaleTool: A Tool for the Accurate Estimation of Technology Scaling in the Deep-Submicron Era. In 2021 IEEE International Symposium on Circuits and Systems (ISCAS). 1-5. https://doi.org/10.1109/ISCAS51556.2021.9401196 · DOI
  69. Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Rita Cuc- chiara. 2022. Retrieval-Augmented Transformer for Image Captioning. In Proceedings of the 19th International Conference on Content-Based Multimedia Indexing (Graz, Austria) (CBMI '22). Association for Computing Machinery, New York, NY, USA, 1-7. https://doi.org/10.1145/3549555.3549585 · DOI
  70. Henry N. Schuh, Arvind Krishnamurthy, David Culler, Henry M. Levy, Luigi Rizzo, Samira Khan, and Brent E. Stephens. 2024. CC-NIC: a Cache-Coherent Interface to the NIC. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 (La Jolla, CA, USA) (ASPLOS '24). Association for Computing Machinery, New York, NY, USA, 52-68. https://doi.org/10.1145/3617232.3624868 · DOI
  71. Haihao Shen, Hanwen Chang, Bo Dong, Yu Luo, and Hengyu Meng. 2023. Efficient llm inference on cpus. arXiv preprint (2023). https://doi.org/10.48550/arXiv.2311.00502 · DOI
  72. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval Augmentation Reduces Hallucination in Conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Compu- tational Linguistics, Punta Cana, Dominican Republic, 3784-3803. https://doi.org/10.18653/v1/2021.findings-emnlp.320 · DOI · Paper page
  73. Joonseop Sim, Soohong Ahn, Taeyoung Ahn, Seungyong Lee, Myunghyun Rhee, Jooyoung Kim, Kwangsik Shin, Donguk Moon, Euiseok Kim, and Kyoung Park. 2023. Computational CXL-Memory Solution for Accelerating Memory-Intensive Ap- plications. IEEE Computer Architecture Letters 22, 1 (2023), 5-8. https://doi.org/10.1109/LCA.2022.3226482 · DOI
  74. Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Inigo Goiri, and Josep Torrellas. 2024. Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference. arXiv preprint arXiv:2403.20306 (2024). arXiv:2403.20306 [cs.AI] https://doi.org/10.48550/arXiv.2403.20306 · DOI
  75. Lisa T. Su, Samuel Naffziger, and Mark Papermaster. 2017. Multi-chip technologies to unleash computing performance gains over the next decade. In 2017 IEEE International Electron Devices Meeting (IEDM). 1.1.1-1.1.8. https://doi.org/10.1109/IEDM.2017.8268306 · DOI
  76. Yixuan Su, David Vandyke, Simon Baker, Yan Wang, and Nigel Collier. 2021. Keep the Primary, Rewrite the Secondary: A Two-Stage Approach for Paraphrase Generation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021,
  77. Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, Online, 560-569. https://doi.org/10.18653/v1/2021.findings-acl.50 · DOI
  78. David Thulke, Nico Daheim, Christian Dugast, and Hermann Ney. 2021. Efficient retrieval augmented generation from unstructured knowledge for task-oriented dialog. arXiv preprint arXiv:2102.04643 (2021). https://doi.org/10.48550/arXiv.2102.04643 · DOI
  79. Zhiliang Tian, Wei Bi, Xiaopeng Li, and Nevin L. Zhang. 2019. Learn- ing to Abstract for Memory-augmented Conversational Response Generation. In Proceedings of the 57th Annual Meeting of the Associ- ation for Computational Linguistics, Anna Korhonen, David Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, Florence, Italy, 3816-3825. https://doi.org/10.18653/v1/P19-1371 · DOI
  80. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, I.
  81. Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. 2021. A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. Pro- ceedings of the VLDB Endowment 14, 11 (jul 2021), 1964-1978. https://doi.org/10.14778/3476249.3476255 · DOI
  82. Yitu Wang, Shiyu Li, Qilin Zheng, Linghao Song, Zongwang Li, Andrew Chang, Hai "Helen" Li, and Yiran Chen. 2024. NDSEARCH: Accelerating Graph-Traversal-Based Approximate Nearest Neigh- bor Search through Near Data Processing. In Proceedings of the 39th Annual International Symposium on Computer Architecture. arXiv:2312.03141 https://doi.org/10.48550/arXiv.2312.03141 · DOI
  83. Zelin Wang, Ping Gong, Yibo Zhang, Jihao Gu, and Xuanyuan Yang. 2023. Retrieval-Augmented Knowledge-Intensive Dialogue. In Natural Language Processing and Chinese Computing, Fei Liu, Nan Duan, Qingting Xu, and Yu Hong (Eds.). Springer Nature Switzerland, Cham, 16-28. https://doi.org/10.48550/arXiv.2005.11401 · DOI · Paper page
  84. Jason Weston, Emily Dinan, and Alexander Miller. 2018. Retrieve and Refine: Improved Sequence Generation Models For Dialogue. In Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI, Aleksandr Chuklin, Jeff Dalton, Julia Kiseleva, Alexey Borisov, and Mikhail Burtsev (Eds.). Association for Computational Linguistics, Brussels, Belgium, 87-92. https://doi.org/10.18653/v1/W18-5713 · DOI · Paper page
  85. Yu Wu, Furu Wei, Shaohan Huang, Yunli Wang, Zhoujun Li, and Ming Zhou. 2019. Response Generation by Context-Aware Prototype Edit- ing. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul. 2019), 7281-7288. https://doi.org/10.1609/aaai.v33i01.33017281 · DOI
  86. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien De- mouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In Inter- national Conference on Machine Learning. PMLR, 38087-38099. https://doi.org/10.5555/3618408.3619993 · DOI
  87. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient Streaming Language Models with Attention Sinks. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=NG7sS51zVF
  88. Jitao Xu, Josep Crego, and Jean Senellart. 2020. Boosting Neural Machine Translation with Similar Translations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Online, 1580-1590. https://doi.org/10.18653/v1/2020.acl-main.144 · DOI
  89. Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. Corrective Retrieval Augmented Generation. arXiv (2024). https://doi.org/10.48550/arXiv.2401.15884 arXiv:2401.15884 [cs.CL] · DOI · Paper page
  90. Nan Yang, Tao Ge, Liang Wang, Binxing Jiao, Daxin Jiang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. Inference with reference: Lossless acceleration of large language models. arXiv preprint arXiv:2304.04487 (2023). https://doi.org/10.48550/arXiv.2304.04487 · DOI
  91. Yifan Yuan, Jinghan Huang, Yan Sun, Tianchen Wang, Jacob Nelson, Dan R. K. Ports, Yipeng Wang, Ren Wang, Charlie Tai, and Nam Sung Kim. 2023. Rambda: RDMA-driven Acceleration Framework for Memory-intensive µs-scale Datacenter Applications. In 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). 499-515. https://doi.org/10.1109/HPCA56546.2023.10071127 · DOI
  92. Hamed Zamani and Michael Bendersky. 2024. Stochastic RAG: End-to- End Retrieval-Augmented Generation through Expected Utility Maxi- mization. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (Washington DC, USA) (SIGIR '24). Association for Computing Machinery, New York, NY, USA, 2641-2646. https://doi.org/10.1145/3626772.3657923 · DOI
  93. Hamed Zamani, Fernando Diaz, Mostafa Dehghani, Donald Metzler, and Michael Bendersky. 2022. Retrieval-Enhanced Machine Learning. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR '22). Association for Computing Machinery, New York, NY, USA, 2875-2886. https://doi.org/10.1145/3477495.3531722 · DOI
  94. Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig, and Satoshi Nakamura. 2018. Guiding Neural Machine Translation with Retrieved Translation Pieces. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Marilyn Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, New Orleans, Louisiana, 1325-1335. https://doi.org/10.18653/v1/N18-1120 · DOI
  95. Yunan Zhang, Shige Liu, and Jianguo Wang. [n. d.]. Are There Funda- mental Limitations in Supporting Vector Data Management in Rela- tional Databases? A Case Study of PostgreSQL. Preprint ([n. d.]). https: //www.cs.purdue.edu/homes/csjgwang/pubs/ICDE24_VecDB.pdf Accepted for publication in Proceedings of the International Conference on Data Engineering (ICDE).
  96. Zhe Zhou, Cong Li, Fan Yang, and Guangyu Sun. 2023. DIMM- Link: Enabling Efficient Inter-DIMM Communication for Near- Memory Processing. In 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). 302-316. https://doi.org/10.1109/HPCA56546.2023.10071005 ISSN: 2378-203X. · DOI
  97. Yun Zhu, Jia-Chen Gu, Caitlin Sikora, Ho Ko, Yinxiao Liu, Chu-Cheng Lin, Lei Shu, Liangchen Luo, Lei Meng, Bang Liu, and Jindong Chen. 2024. Accelerating Inference of Retrieval- Augmented Generation via Sparse Context Selection. arXiv (2024). https://doi.org/10.48550/arXiv.2405.16178 · DOI

Cited by

Sorted by citations

Authors
Quinn, D., Nouri, M., Patel, N., Salihu, J., Salemi, A., Lee, S., Zamani, H., & Alian, M.
Affiliations
  • Cornell University (US)
  • University of Kansas (US)
  • University of Massachusetts Amherst (US)
  • Samsung Electronics (South Korea) (KR)
ORCID
Venue
Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1
Publisher
ACM
Published
2025-03-30
Pages
15-32
DOI
10.1145/3669940.3707264
OpenAlex
W4407197060
Cited by
48
References
8
Citations by year
2026: 24 · 2025: 24
FWCI
33.72
Type
conference paper
Language
en
Open access
Licence
Locations
  • International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Office of Naval Research (N000142412612)
  • National Science Foundation (2239020,1565570,2402873)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
Keywords
retrieval-augmented generation, large language models, hallucinations, approximate nearest neighbor search, inference acceleration, retrieval models, document retrieval, web search

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review