ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
Open accessAbstract
Jon Saad-Falcon, Omar Khattab, Christopher Potts, Matei Zaharia. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Standard RAG evaluation frameworks (RAGAS faithfulness, FActScore, ARES, FaithDial) all pass deceptive grounding responses by design because every claim is sourced from a real document.
“Standard RAG evaluation frameworks (RAGAS faithfulness [Es et al., 2024], FActScore [Min et al., 2023], ARES [Saad-Falcon et al., 2024], FaithDial [Dziri et al., 2022]) all pass deceptive grounding responses by design because every claim is sourced from a real document.”
From Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation (2026) · cites this paper as Saad-Falcon et al., 2024
The ARES description says its answer-faithfulness judge checks whether an answer is grounded in the retrieved document, which supports the claim’s description of one evaluation criterion. But the paper also describes context- and answer-relevance checks, and neither those passages nor the AIS results say that ARES passes deceptive grounding responses by design or merely because claims come from real documents. This verdict concerns ARES, not the other cited frameworks. (LLM, conf=0.82)
Experimental — accuracy not yet measuredRAG evaluation frameworks such as RAGAS and ARES assess faithfulness at the answer level, treating the full context-answer pair as a unit rather than scoring individual attribution-citation pairs.
“RAG evaluation frameworks such as RAGAS [Es et al., 2024] and ARES [Saad-Falcon et al., 2024] assess faithfulness at the answer level, treating the full context-answer pair as a unit rather than scoring individual attribution-citation pairs”
From Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution (2026) · cites this paper as Saad-Falcon et al., 2024
No existing framework implements entity-attribution verification.
“no existing framework implements it.”
From Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation (2026) · cites this paper as Saad-Falcon et al., 2024
The Abstract and the section on ARES results on the AIS benchmark describe evaluating answer faithfulness and whether generated answers are attributed to retrieved passages. Neither addresses entity-attribution verification specifically or says that no existing framework implements it. Searched the indexed copy for “entity-attribution verification” and related terms; the passages provide no page numbers. (LLM, conf=0.73)
Experimental — accuracy not yet measuredRAG systems are evaluated along context relevance, answer faithfulness, and answer relevance.
“Consequently, RAG systems are evaluated based on dimensions such as context relevance (that is, whether the retrieved context is relevant to the given prompt), answer faithfulness (that is, whether the response generated by the LLM is properly grounded in the retrieved context), and answer relevance (that is, whether the response is relevant to the user question) [27,100].”
From Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) (2024) · cites this paper as [100]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 11
- Mubashara Akhtar, Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, and Andreas Vlachos, editors. 2023. Proceedings of the Sixth Fact Extraction and VERification Workshop (FEVER). Association for Computational Linguistics, Dubrovnik, Croatia.
- Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Lafor- est, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
- Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International confer- ence on machine learning, pages 3929-3938. PMLR.
- Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535-547.
- Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface: A challenge set for reading com- prehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa- pers), pages 252-262.
- Omar Khattab, Christopher Potts, and Matei Zaharia. 2021. Relevance-guided supervision for openqa with colbert. Transactions of the association for computa- tional linguistics, 9:929-944.
- Diederik P. Kingma and Jimmy Ba. 2017. Adam: A method for stochastic optimization. Tom Kocmi and Christian Federmann. 2023. Large language models are state-of-the-art evaluators of translation quality. arXiv preprint arXiv:2302.14520. · arXiv
- Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, and Kyle Lo. 2023. LongEval: Guidelines for human evaluation of faithfulness in long-form summarization. In Proceed- ings of the 17th Conference of the European Chap- ter of the Association for Computational Linguistics, pages 1650-1669, Dubrovnik, Croatia. Association for Computational Linguistics.
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453- 466.
- David P Sander and Laura Dietz. 2021. Exam: How to evaluate retrieve-and-generate systems for users who do not (yet) know what they want. In DESIRES, pages 136-146.
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Aman- preet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stick- ier benchmark for general-purpose language under- standing systems. Advances in neural information processing systems, 32.
Cited by
Sorted by citations
A comprehensive survey of loss functions and metrics in deep learning
Terven, J. R., Esparza, D. M. C., Romero-González, J.-A., Ramírez-Pedraza, A., & Chávez‐Urbiola, E. A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11198-7 · Cited by 517 · see citing works
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2026) · Data Science and Engineering · doi:10.1007/s41019-025-00335-5 · Cited by 549 · see citing works
Lyu, Y., li, Z., Niu, S., Xiong, F., Tang, B., Wang, W., Wu, H., Liu, H., Xu, T., & Chen, E. (2024) · ACM Transactions on Information Systems · doi:10.1145/3701228 · Cited by 144 · see citing works
When Search Engine Services Meet Large Language Models: Visions and Challenges
Xiong, H., Bian, J., Li, Y., Li, X., Du, M., Wang, S., Yin, D., & Helal, S. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3451185 · Cited by 82 · see citing works
Hashemi, H., Eisner, J., Rosset, C., Van Durme, B., & Kedzie, C. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.745 · Cited by 47 · see citing works
Evaluating Retrieval-Augmented Generation Models for Financial Report Question and Answering
Iaroshev, I., Pillai, R., Vaglietti, L., & Hanne, T. (2024) · Applied Sciences · doi:10.3390/app14209318 · Cited by 45 · see citing works
Hybrid Retrieval-Augmented Generation Approach for LLMs Query Response Enhancement
Omrani, P., Hosseini, A., Hooshanfar, K., Ebrahimian, Z., Toosi, R., & Akhaee, M. A. (2024) · International Conference on Web Research (ICWR) · doi:10.1109/icwr61162.2024.10533345 · Cited by 44 · see citing works
Kenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36 · see citing works
Emerging trends: a gentle introduction to RAG
Church, K., Sun, J., Yue, R., Vickers, P., Saba, W., & Chandrasekar, R. (2024) · Natural Language Engineering · doi:10.1017/s1351324924000044 · Cited by 32 · see citing works
Karakurt, E., & Akbulut, A. (2025) · Applied Sciences · doi:10.3390/app16010368 · Cited by 39 · see citing works
Correctness is not Faithfulness in Retrieval Augmented Generation Attributions
Wallat, J., Heuss, M., Rijke, M. D., & Anand, A. (2025) · Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) · doi:10.1145/3731120.3744592 · Cited by 21 · see citing works
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Mitra, B., Cramer, H. S. M., & Gurevich, O. (2024) · The information retrieval series · doi:10.1007/978-3-031-73147-1_7 · Cited by 20 · see citing works
The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Models
Pradeep, R., Thakur, N., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I., Dang, H. T., & Lin, J. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3730090 · Cited by 14 · see citing works
Securing RAG: A Risk Assessment and Mitigation Framework
Ammann, L., Ott, S., Landolt, C. R., & Lehmann, M. P. (2025) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds66131.2025.00024 · Cited by 14 · see citing works
Can LLMs be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets
Brehme, L., Ströhle, T., & Breu, R. (2025) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds66131.2025.00010 · Cited by 13 · see citing works
Conversational Gold: Evaluating Personalized Conversational Search System Using Gold Nuggets
Abbasiantaeb, Z., Lupart, S., Azzopardi, L. A., Dalton, J., & Aliannejadi, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730316 · Cited by 15 · see citing works
Towards Efficient Educational Chatbots: Benchmarking RAG Frameworks
Khan, U. A., Khan, F., Khan, E., Hasnain, M. A., & Moinuddin, A. A. (2025) · International Conference on Emerging Technologies in Computing and Communication (ETCC) · doi:10.1109/etcc65847.2025.11108323 · Cited by 9 · see citing works
Ahmad, S. N., Nezami, Z., Hafeez, M., & Zaidi, S. A. R. (2025) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc62392.2025.11274810 · Cited by 9 · see citing works
A Survey of Retrieval-Augmented Generation (RAG) for Large Language Models
Ma, Y., Nie, H., Chen, C., Zhang, J., Jiang, J., Wang, B., & Xia, Y. (2025) · International Conference on Trustworthy Big Data and Artificial Intelligence (ICTBAI) · doi:10.1109/ictbai68361.2025.00008 · Cited by 9 · see citing works
Using AI to assess corporate climate transition disclosures
Senni, C. C., Schimanski, T., Bingler, J., Ni, J., & Leippold, M. (2024) · Environmental Research Communications · doi:10.1088/2515-7620/ad9e88 · Cited by 8 · see citing works
Evaluating Open-Source LLMs in RAG Systems: A Benchmark on Diploma Theses Abstracts Using Ragas
Antal, M., & Búza, K. (2025) · Acta Universitatis Sapientiae Informatica · doi:10.1007/s44427-025-00006-3 · Cited by 9 · see citing works
Unlocking Multi-View Insights in Knowledge-Dense Retrieval-Augmented Generation
Chen, G., Yu, W., Lu, X., Zhang, X., Meng, E., & Sha, L. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3622944 · Cited by 8 · see citing works
Rag and Rau: A Survey on Retrieval-Augmented Language Model in Natural Language ProcessingPreprint
Hu, Y., & Lu, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5015182 · Cited by 8 · see citing works
The Viability of Crowdsourcing for RAG Evaluation
Gienapp, L., Hagen, T., Fröbe, M., Hagen, M., Stein, B., Potthast, M., & Scells, H. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730093 · Cited by 7 · see citing works
Kurland, J., Varadharaju, V., Liu, A., Stokes, P., Gupta, A., Hudspeth, M., & O'Connor, B. (2025) · American Journal of Speech-Language Pathology · doi:10.1044/2025_ajslp-24-00400 · Cited by 6 · see citing works
RAG Powered LLMs for QA: Evolution, Challenges, Applications, and Future Directions
Zeeshan, H. M. A., Faizan, M., Zia, U., & Gohar, A. (2025) · International Conference on Communication Technologies (ComTech) · doi:10.1109/comtech65062.2025.11034531 · Cited by 6 · see citing works
Thakur, N., Pradeep, R., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I. M., Dang, H. T., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730165 · Cited by 6 · see citing works
RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation
Cheng, S., Li, J., Wang, H., & Ma, Y. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747741 · Cited by 7 · see citing works
Packowski, S., Halilovic, I., Schlotfeldt, J., & Smith, T. (2024) · International Conference on Advances in Artificial Intelligence (ICAAI) · doi:10.1145/3704137.3704181 · Cited by 11 · see citing works
Evaluating Precision and Recall at Retrieval Time in Retrieval-Augmented Generation (RAG) Systems
Agnihotram, G., & Sarkar, J. (2025) · American Journal of Computer Science and Technology · doi:10.11648/j.ajcst.20250804.11 · Cited by 5 · see citing works
Tao, J., Pavlick, E., Grondin, A., Bustamante, J. D., Martin, H., Parent, H., Fenn, N. Q., Almonte, A. A., Maguire-Wilkerson, A., Gu, M., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/79671 · Cited by 5 · see citing works
Ye, Z., Qi, D., Liu, H., & Zhang, S. (2025) · Electronics · doi:10.3390/electronics14163284 · Cited by 5 · see citing works
Evaluation of Orca 2 Against Other LLMs for Retrieval Augmented Generation
Huang, D., & Wang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2650-9_1 · Cited by 5 · see citing works
Sheik, R., Reji, S. A., Sharon, A., Avisha, M., & Nirmala, S. J. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09459-5 · Cited by 5 · see citing works
Kartiyanta, M. A., Ancilla, E., & Jingga, K. (2025) · IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) · doi:10.1109/iaict65714.2025.11101522 · Cited by 4 · see citing works
Knollmeyer, S., Caymazer, O., Koval, L., Akmal, M. U., Asif, S., Mathias, S. G., & Großmann, D. (2026) · SN Computer Science · doi:10.1007/s42979-026-05134-x · Cited by 6 · see citing works
ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate DisclosuresPreprint
Schimanski, T., Ni, J., Martin, R. S., Ranger, N. A., & Leippold, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4866498 · Cited by 3 · see citing works
Avula, S., Lee, C., Zhang, R., & Murdock, V. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730230 · Cited by 2 · see citing works
Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality
Feng, N., Sui, Y., Hou, S., Cresswell, J. C., & Wu, G. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730244 · Cited by 3 · see citing works
LLM4Eval: Large Language Model for Evaluation in IR
Siro, C., Rahmani, H. A., Aliannejadi, M., Craswell, N., Clarke, C. L. A., Faggioli, G., Mitra, B., Thomas, P., & Yılmaz, E. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730367 · Cited by 2 · see citing works
Axioms for Retrieval-Augmented Generation
Reimer, J. H., Fröbe, M., Stein, B., Potthast, M., & Hagen, M. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744601 · Cited by 2 · see citing works
Jin, Q., Wang, L., Yuan, W., & Chang, D. (2026) · Journal of Engineering Design · doi:10.1080/09544828.2026.2639933 · Cited by 3 · see citing works
TrueLens: Video Fake News Detection with Dual Level Evidence Gathering and Consolidation
Chen, J., Liu, Q., Sun, J., & Zhang, Y. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792362 · Cited by 2 · see citing works
Zhao, Y., Miao, Y., Guo, R., Luo, Y., Wang, H., & Wu, Y. X. (2026) · Journal of Medical Internet Research · doi:10.2196/90046 · Cited by 2 · see citing works
CoURAGE: A Framework to Evaluate RAG Systems
Galla, D., Hoda, S., Zhang, M., Quan, W., Yang, T. D., & Voyles, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70242-6_37 · Cited by 2 · see citing works
Knoll: Creating a Knowledge Ecosystem for Large Language Models
Zhao, D., Yang, D., & Bernstein, M. S. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747711 · Cited by 2 · see citing works
From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
Ahn, J., & Kim, M. (2026) · arXiv · doi:10.48550/arxiv.2607.08028 · Cited by 1 · see citing works
DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research
Coelho, J., Ning, J., He, J., Mao, K., Paladugu, A., Setlur, P., Jin, J., Callan, J., Magalhães, J., Martins, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.19253 · Cited by 1 · see citing works
In-Context Learning for Scalable and Online Hallucination Detection in RAGS
Albanese, N. C. (2024) · doi:10.5121/csit.2024.140703 · Cited by 1 · see citing works
Wang, B., Bai, Z., Wang, Q., Lin, J., Yang, M., Zeng, X., & Xu, R. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730232 · Cited by 1 · see citing works
AdmPModeler: Modeling Administrative Processes Using Large Language Models. A Case Study
Macrì, M., Luzi, F. D., & Mecella, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-01589-1_19 · Cited by 1 · see citing works
ConSens: Assessing Context Grounding in Open-Book Question Answering
Vankov, I., Ivanov, M., Correia, A. D., & Botev, V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04549-2_13 · Cited by 1 · see citing works
Wang, Y., Zhang, Y., Xu, K., Xu, Y., Wang, Y., & Feng, Q. (2025) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai65422.2025.11189813 · Cited by 1 · see citing works
Blachnik, M., & Chmielewski, J. (2025) · Annals of Computer Science and Information Systems · doi:10.15439/2025f6165 · Cited by 1 · see citing works
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 1 · see citing works
EduX-RAG: Retrieval Augmented Generation Framework for Cross-Lingual Educational Chatbots
Medina, J., Jung, S. G., Salminen, J. O., Aldous, K. K., & Jansen, B. J. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11390969 · Cited by 1 · see citing works
Wei, Y., Li, X., Huang, Z., Lin, Z., & Zhang, N. (2026) · Computers in Industry · doi:10.1016/j.compind.2026.104462 · Cited by 1 · see citing works
UniEval-RAG: A Unified End-to-End Evaluation Framework for RAG Systems
Liang, S., Gong, X., Li, F., Zhou, H., Wu, Y., & Liao, C. (2025) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1109/ciotsc67482.2025.11412981 · Cited by 1 · see citing works
Guainazzo, N., Delzanno, G., Ancona, D., & D’Agostino, D. (2026) · Sensors · doi:10.3390/s26051590 · Cited by 1 · see citing works
When Semantic Similarity Fails: Analyzing Metric Divergence in Legal RAG Evaluation
SILVA, E. D. D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21324-2_43 · Cited by 1 · see citing works
FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG
Dassen, M., Kotula, R., Murray, K., Yates, A., Lawrie, D., Kayi, E. S., Mayfield, J., & Duh, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_18 · Cited by 3 · see citing works
Sugimoto, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/406042 · Cited by 1 · see citing works
The Development of Information Retrieval Model for Hadith Data from Knowledge Graph
Norhisham, N. N., Muhammad, N., Ismail, M. I. F., & Fabregas, A. C. (2026) · International Conference on Electronics, Computer and Computation · doi:10.1109/icecco67619.2026.11488814 · Cited by 1 · see citing works
Can We Evaluate RAGs with Synthetic Data?
Elburg, J. M. V., Putten, P. V. D., & Marx, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19102-1_11 · Cited by 2 · see citing works
Burillo, J. D., Neira, R. B. H., & Borrás-Gené, O. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10090299 · Cited by 1 · see citing works
Schnieder, M., Schimanke, F., Sert, M., Persia, F., Lyu, C., & Glesener, G. B. (2026) · International Journal of Semantic Computing · doi:10.1142/s1793351x26430038 · Cited by 1 · see citing works
Faridi, A. R., Masood, F., Keshvi, & Yunus, T. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70122 · Cited by 2 · see citing works
Tolsgaard, M. G., Grierson, L., Pusic, M. V., & Turner, L. (2026) · Medical Education · doi:10.1111/medu.70309 · Cited by 2 · see citing works
Kawashima, S., Shiramatsu, S., & Mizumoto, T. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3305-7_48 · Cited by 1 · see citing works
Avaliação do Impacto de Diferentes Padrões Arquiteturais RAG em Domínios Jurídicos
Paranhos, S. L., Tomazini, J. N., Camilo-Junior, C. G., & de Oliveira, S. S. T. (2024) · Escola Regional de Informática de Goiás (ERI-GO) · doi:10.5753/erigo.2024.4846 · Cited by 1 · see citing works
Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
Caruzzo, C., Yoo, D., & Kim, T. S. (2026) · arXiv · doi:10.48550/arxiv.2607.09349
Same Ranking, Different Winner: How Scoring Targets Shape LLM Memory Benchmarks
Panthi, S., & Abdelfattah, R. (2026) · arXiv · doi:10.48550/arxiv.2605.24060
Leung, E., Lumer, E., Feld, C., Huber, A., Subbiah, V. K., & Paul, K. (2026) · arXiv · doi:10.48550/arxiv.2607.08700
DeepResearchGym: A Free, Transparent, and Reproducible Sandbox for Deep Research
Coelho, J., Ning, J., He, J., Mao, K., Paladugu, A. S., Setlur, P., Jin, J., Callan, J., Magalhães, J., Martins, B., et al. (2026) · Proceedings of the 2026 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) · doi:10.1145/3805713.3820409
Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs
Belikova, J., Polev, K., Parchiev, R., & Simakov, D. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731969
Liu, Y., Yin, Z., Wang, J., & Li, K. (2025) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc65474.2025.11090835
Automated Question-Answer Generation for Evaluating Vietnamese RAG-Based Approaches
Nguyen, D.-H., & Nguyen, L. H. B. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-98164-7_21
Kim, B., & Yang, J. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-3141-7_13
RagPharma: A RAG-Based Chatbot for Medicine Leaflets with a Dual-Dataset Evaluation Framework
Navarro, L. C., Mutz, F. W., Paixão, T. M., Zanetti, G. G., Badue, C. S., Souza, A. F. D., & Oliveira-Santos, T. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5767
Reference-Aligned Retrieval-Augmented Question Answering over Heterogeneous Proprietary Documents
Choi, N., Byun, G., Chung, A., Paek, E. S., Lee, S., & Choi, J. D. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761566
TechSupportEval: An Automated Evaluation Framework for Technical Support Question Answering
Chen, B., Sun, Y., Liu, Y., Xu, L., Xie, Z., Pei, C., Han, J., Ni, F., Cai, X., Yang, C., et al. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227545
Benchmarking Retrieval-Augmented Generation: Challenging Domain-specific Datasets Generation Method
Liu, Y., Hu, L., Dong, X., & Wang, Z. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228599
Applying Large Language Models to Search
Benedetti, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01563-1_4
Enhancing Academic Search with Agentic RAG and Knowledge Graphs: A Case Study on EconBiz
Hajra, A. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10721-3_77
ChatAcadien: A RAG-LLM-Based Chatbot for Exploring Acadian Genealogy
Ghali, R., & Selouani, S. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00026 · Cited by 1 · see citing works
PRECISE: Reducing the bias of LLM evaluations using prediction‐powered ranking estimation
Divekar, A., & Majumder, A. (2026) · AI Magazine · doi:10.1002/aaai.70066
Liu, Y., Wang, Z., Chen, Y., He, Y., Lian, J., & Lu, H. (2025) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc67331.2025.00010
Relation-Aware Retrieval Augmented Generation for Relation Extraction
Tian, L., Bu, C., Huang, M., & Wu, X. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342441
Explainable Knowledge Access: Recursive and Rerank-Based RAG for Interpretable QA
Cosseddu, M., Giuliani, A., Manca, M. M., Pilloni, M., Podda, A. S., & Tiddia, S. G. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-15638-9_13
Nikolić, M., Stanimirović, A., & Stoimenov, L. V. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-04890-5_32
RAG-CUE: Actionable and Unified Evaluation for Retrieval-Augmented Generation Systems
Prabhakar, P. K., Shukla, N. K., & Krishnamurthy, V. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401780
Dobrovodsky, P., Eszter, B. D. K., Tafferner-Gulyás, V., & Fleiner, R. (2026) · IEEE World Symposium on Applied Machine Intelligence and Informatics (SAMI) · doi:10.1109/sami68106.2026.11420463
Talmud-IR: A Talmud-Inspired Interface for Discussing RAG Response Quality
Kusa, W., Deckers, N., Fröbe, M., Dietz, L., Platow, B., & Sanderson, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_21
Zhang, J., He, R., Zheng, X., Lin, Y., Lu, S., & Qu, Y. (2025) · International Symposium on Artificial Intelligence and Big Data (AIBDF) · doi:10.1109/aibdf67964.2025.11440704
A Graph-Based RAG System for Enhanced Information Gathering in Local Newsrooms
Pillai, R. G., Fokkens, A. S., & van Atteveldt, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25159-6_25
Persistent and Conversational Multi-Method Explainability for Trustworthy Financial AI
Makridis, G., Fatouros, G., Soldatos, J., Katsis, G., & Kyriazis, D. (2026) · International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) · doi:10.1109/dcoss-iot69657.2026.00158
Beyond the Rules: Understanding the Design Logic of Internet Standards
Bian, J., Welzl, M., & Arefyev, N. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795082
Explainable academic advising via category-routed knowledge graph retrieval and generation
Nguyen, H.-H., La, T.-T., & Ma, T.-T. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07300-3
TCK-QA: Benchmarking Knowledge Graph-Augmented LLMs for Question Answering on the Turkish Penal CodePreprint
Ay, B., & Okuyan, H. (2026) · Research Square · doi:10.21203/rs.3.rs-9804227/v1
Xu, Z., Wang, Z., Wang, H., Zhou, Y., & Sun, M. (2026) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit70033.2026.11558060
- Authors
- Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Venue
- Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 338-354
- OpenAlex
- W4401042808
- Cited by
- 148
- References
- 0
- Citations by year
- 2026: 84 · 2025: 48 · 2024: 16
- FWCI
- 43.61
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- retrieval-augmented generation, automated assessment, evaluation framework, large language models, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
