The DOI matches its Crossref record; 17 of the 20 references it cites were found in a registry or an index.
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
Open accessAbstract
Jon Saad-Falcon, Omar Khattab, Christopher Potts, Matei Zaharia. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Standard RAG evaluation frameworks (RAGAS faithfulness [Es et al., 2024], FActScore [Min et al., 2023], ARES [Saad-Falcon et al., 2024], FaithDial [Dziri et al., 2022]) all pass deceptive grounding responses by design because every claim is sourced from a real document.”
From Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation (2026) · cites this paper as Saad-Falcon et al., 2024
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“RAG evaluation frameworks such as RAGAS [Es et al., 2024] and ARES [Saad-Falcon et al., 2024] assess faithfulness at the answer level, treating the full context-answer pair as a unit rather than scoring individual attribution-citation pairs”
From Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution (2026) · cites this paper as Saad-Falcon et al., 2024
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“no existing framework implements it.”
From Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation (2026) · cites this paper as Saad-Falcon et al., 2024
Experimental — accuracy not yet measuredIn the citing paper
“Consequently, RAG systems are evaluated based on dimensions such as context relevance (that is, whether the retrieved context is relevant to the given prompt), answer faithfulness (that is, whether the response generated by the LLM is properly grounded in the retrieved context), and answer relevance (that is, whether the response is relevant to the user question) [27,100].”
From Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) (2024) · cites this paper as [100]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 11
- Mubashara Akhtar, Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, and Andreas Vlachos, editors. 2023. Proceedings of the Sixth Fact Extraction and VERification Workshop (FEVER). Association for Computational Linguistics, Dubrovnik, Croatia.
- Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Lafor- est, and Elena Simperl. 2018. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
- Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International confer- ence on machine learning, pages 3929-3938. PMLR.
- Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535-547.
- Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface: A challenge set for reading com- prehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa- pers), pages 252-262.
- Omar Khattab, Christopher Potts, and Matei Zaharia. 2021. Relevance-guided supervision for openqa with colbert. Transactions of the association for computa- tional linguistics, 9:929-944.
- Diederik P. Kingma and Jimmy Ba. 2017. Adam: A method for stochastic optimization. Tom Kocmi and Christian Federmann. 2023. Large language models are state-of-the-art evaluators of translation quality. arXiv preprint arXiv:2302.14520. · arXiv
- Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, and Kyle Lo. 2023. LongEval: Guidelines for human evaluation of faithfulness in long-form summarization. In Proceed- ings of the 17th Conference of the European Chap- ter of the Association for Computational Linguistics, pages 1650-1669, Dubrovnik, Croatia. Association for Computational Linguistics.
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453- 466.
- David P Sander and Laura Dietz. 2021. Exam: How to evaluate retrieve-and-generate systems for users who do not (yet) know what they want. In DESIRES, pages 136-146.
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Aman- preet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stick- ier benchmark for general-purpose language under- standing systems. Advances in neural information processing systems, 32.
Cited by
Sorted by citations
A comprehensive survey of loss functions and metrics in deep learning
Terven, J. R., Esparza, D. M. C., Romero-González, J.-A., Ramírez-Pedraza, A., & Chávez‐Urbiola, E. A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11198-7 · Cited by 685
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2026) · Data Science and Engineering · doi:10.1007/s41019-025-00335-5 · Cited by 871
Lyu, Y., li, Z., Niu, S., Xiong, F., Tang, B., Wang, W., Wu, H., Liu, H., Xu, T., & Chen, E. (2024) · ACM Transactions on Information Systems · doi:10.1145/3701228 · Cited by 199
When Search Engine Services Meet Large Language Models: Visions and Challenges
Xiong, H., Bian, J., Li, Y., Li, X., Du, M., Wang, S., Yin, D., & Helal, S. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3451185 · Cited by 111
Hashemi, H., Eisner, J., Rosset, C., Van Durme, B., & Kedzie, C. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.745 · Cited by 47
Evaluating Retrieval-Augmented Generation Models for Financial Report Question and Answering
Iaroshev, I., Pillai, R., Vaglietti, L., & Hanne, T. (2024) · Applied Sciences · doi:10.3390/app14209318 · Cited by 60
Hybrid Retrieval-Augmented Generation Approach for LLMs Query Response Enhancement
Omrani, P., Hosseini, A., Hooshanfar, K., Ebrahimian, Z., Toosi, R., & Akhaee, M. A. (2024) · International Conference on Web Research (ICWR) · doi:10.1109/icwr61162.2024.10533345 · Cited by 48
Kenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data
Dorner, F. E., Nastl, V. Y., & Hardt, M. (2024) · arXiv · doi:10.48550/arxiv.2410.13341 · Cited by 31
Emerging trends: a gentle introduction to RAG
Church, K., Sun, J., Yue, R., Vickers, P., Saba, W., & Chandrasekar, R. (2024) · Natural Language Engineering · doi:10.1017/s1351324924000044 · Cited by 33
Karakurt, E., & Akbulut, A. (2025) · Applied Sciences · doi:10.3390/app16010368 · Cited by 45
Correctness is not Faithfulness in Retrieval Augmented Generation Attributions
Wallat, J., Heuss, M., Rijke, M. D., & Anand, A. (2025) · Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) · doi:10.1145/3731120.3744592 · Cited by 21
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Mitra, B., Cramer, H. S. M., & Gurevich, O. (2024) · The information retrieval series · doi:10.1007/978-3-031-73147-1_7 · Cited by 28
The Great Nugget Recall: Automating Fact Extraction and RAG Evaluation with Large Language Models
Pradeep, R., Thakur, N., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I., Dang, H. T., & Lin, J. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3730090 · Cited by 14
Securing RAG: A Risk Assessment and Mitigation Framework
Ammann, L., Ott, S., Landolt, C. R., & Lehmann, M. P. (2025) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds66131.2025.00024 · Cited by 24
Can LLMs be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets
Brehme, L., Ströhle, T., & Breu, R. (2025) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds66131.2025.00010 · Cited by 21
Conversational Gold: Evaluating Personalized Conversational Search System Using Gold Nuggets
Abbasiantaeb, Z., Lupart, S., Azzopardi, L. A., Dalton, J., & Aliannejadi, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730316 · Cited by 15
Towards Efficient Educational Chatbots: Benchmarking RAG Frameworks
Khan, U. A., Khan, F., Khan, E., Hasnain, M. A., & Moinuddin, A. A. (2025) · International Conference on Emerging Technologies in Computing and Communication (ETCC) · doi:10.1109/etcc65847.2025.11108323 · Cited by 8
Ahmad, S. N., Nezami, Z., Hafeez, M., & Zaidi, S. A. R. (2025) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc62392.2025.11274810 · Cited by 9
A Survey of Retrieval-Augmented Generation (RAG) for Large Language Models
Ma, Y., Nie, H., Chen, C., Zhang, J., Jiang, J., Wang, B., & Xia, Y. (2025) · International Conference on Trustworthy Big Data and Artificial Intelligence (ICTBAI) · doi:10.1109/ictbai68361.2025.00008 · Cited by 9
Using AI to assess corporate climate transition disclosures
Senni, C. C., Schimanski, T., Bingler, J., Ni, J., & Leippold, M. (2024) · Environmental Research Communications · doi:10.1088/2515-7620/ad9e88 · Cited by 21
Evaluating Open-Source LLMs in RAG Systems: A Benchmark on Diploma Theses Abstracts Using Ragas
Antal, M., & Búza, K. (2025) · Acta Universitatis Sapientiae Informatica · doi:10.1007/s44427-025-00006-3 · Cited by 9
Unlocking Multi-View Insights in Knowledge-Dense Retrieval-Augmented Generation
Chen, G., Yu, W., Lu, X., Zhang, X., Meng, E., & Sha, L. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3622944 · Cited by 8
Rag and Rau: A Survey on Retrieval-Augmented Language Model in Natural Language ProcessingPreprint
Hu, Y., & Lu, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5015182 · Cited by 8
The Viability of Crowdsourcing for RAG Evaluation
Gienapp, L., Hagen, T., Fröbe, M., Hagen, M., Stein, B., Potthast, M., & Scells, H. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730093 · Cited by 7
Kurland, J., Varadharaju, V., Liu, A., Stokes, P., Gupta, A., Hudspeth, M., & O'Connor, B. (2025) · American Journal of Speech-Language Pathology · doi:10.1044/2025_ajslp-24-00400 · Cited by 6
RAG Powered LLMs for QA: Evolution, Challenges, Applications, and Future Directions
Zeeshan, H. M. A., Faizan, M., Zia, U., & Gohar, A. (2025) · International Conference on Communication Technologies (ComTech) · doi:10.1109/comtech65062.2025.11034531 · Cited by 6
Thakur, N., Pradeep, R., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I. M., Dang, H. T., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730165 · Cited by 6
RAGTrace: Understanding and Refining Retrieval-Generation Dynamics in Retrieval-Augmented Generation
Cheng, S., Li, J., Wang, H., & Ma, Y. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747741 · Cited by 10
Packowski, S., Halilovic, I., Schlotfeldt, J., & Smith, T. (2024) · International Conference on Advances in Artificial Intelligence (ICAAI) · doi:10.1145/3704137.3704181 · Cited by 16
Evaluating Precision and Recall at Retrieval Time in Retrieval-Augmented Generation (RAG) Systems
Agnihotram, G., & Sarkar, J. (2025) · American Journal of Computer Science and Technology · doi:10.11648/j.ajcst.20250804.11 · Cited by 5
Tao, J., Pavlick, E., Grondin, A., Bustamante, J. D., Martin, H., Parent, H., Fenn, N. Q., Almonte, A. A., Maguire-Wilkerson, A., Gu, M., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/79671 · Cited by 5
Ye, Z., Qi, D., Liu, H., & Zhang, S. (2025) · Electronics · doi:10.3390/electronics14163284 · Cited by 6
Evaluation of Orca 2 Against Other LLMs for Retrieval Augmented Generation
Huang, D., & Wang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2650-9_1 · Cited by 5
Sheik, R., Reji, S. A., Sharon, A., Avisha, M., & Nirmala, S. J. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09459-5 · Cited by 5
Kartiyanta, M. A., Ancilla, E., & Jingga, K. (2025) · IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) · doi:10.1109/iaict65714.2025.11101522 · Cited by 6
Knollmeyer, S., Caymazer, O., Koval, L., Akmal, M. U., Asif, S., Mathias, S. G., & Großmann, D. (2026) · SN Computer Science · doi:10.1007/s42979-026-05134-x · Cited by 9
ClimRetrieve: A Benchmarking Dataset for Information Retrieval from Corporate Climate DisclosuresPreprint
Schimanski, T., Ni, J., Martin, R. S., Ranger, N. A., & Leippold, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4866498 · Cited by 4
Avula, S., Lee, C., Zhang, R., & Murdock, V. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730230 · Cited by 3
Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality
Feng, N., Sui, Y., Hou, S., Cresswell, J. C., & Wu, G. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730244 · Cited by 8
LLM4Eval: Large Language Model for Evaluation in IR
Siro, C., Rahmani, H. A., Aliannejadi, M., Craswell, N., Clarke, C. L. A., Faggioli, G., Mitra, B., Thomas, P., & Yılmaz, E. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730367 · Cited by 2
Axioms for Retrieval-Augmented Generation
Reimer, J. H., Fröbe, M., Stein, B., Potthast, M., & Hagen, M. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744601 · Cited by 2
Jin, Q., Wang, L., Yuan, W., & Chang, D. (2026) · Journal of Engineering Design · doi:10.1080/09544828.2026.2639933 · Cited by 3
TrueLens: Video Fake News Detection with Dual Level Evidence Gathering and Consolidation
Chen, J., Liu, Q., Sun, J., & Zhang, Y. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792362 · Cited by 2
Zhao, Y., Miao, Y., Guo, R., Luo, Y., Wang, H., & Wu, Y. X. (2026) · Journal of Medical Internet Research · doi:10.2196/90046 · Cited by 2
CoURAGE: A Framework to Evaluate RAG Systems
Galla, D., Hoda, S., Zhang, M., Quan, W., Yang, T. D., & Voyles, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70242-6_37 · Cited by 2
Knoll: Creating a Knowledge Ecosystem for Large Language Models
Zhao, D., Yang, D., & Bernstein, M. S. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747711 · Cited by 7
From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
Ahn, J., & Kim, M. (2026) · arXiv · doi:10.48550/arxiv.2607.08028 · Cited by 1
DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research
Coelho, J., Ning, J., He, J., Mao, K., Paladugu, A., Setlur, P., Jin, J., Callan, J., Magalhães, J., Martins, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.19253 · Cited by 1
In-Context Learning for Scalable and Online Hallucination Detection in RAGS
Albanese, N. C. (2024) · doi:10.5121/csit.2024.140703 · Cited by 1
Wang, B., Bai, Z., Wang, Q., Lin, J., Yang, M., Zeng, X., & Xu, R. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730232 · Cited by 1
AdmPModeler: Modeling Administrative Processes Using Large Language Models. A Case Study
Macrì, M., Luzi, F. D., & Mecella, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-01589-1_19 · Cited by 1
ConSens: Assessing Context Grounding in Open-Book Question Answering
Vankov, I., Ivanov, M., Correia, A. D., & Botev, V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04549-2_13 · Cited by 1
Wang, Y., Zhang, Y., Xu, K., Xu, Y., Wang, Y., & Feng, Q. (2025) · International Conference on Computer Communication and Artificial Intelligence (CCAI) · doi:10.1109/ccai65422.2025.11189813 · Cited by 1
Blachnik, M., & Chmielewski, J. (2025) · Annals of Computer Science and Information Systems · doi:10.15439/2025f6165 · Cited by 1
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 1
EduX-RAG: Retrieval Augmented Generation Framework for Cross-Lingual Educational Chatbots
Medina, J., Jung, S. G., Salminen, J. O., Aldous, K. K., & Jansen, B. J. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11390969 · Cited by 1
Wei, Y., Li, X., Huang, Z., Lin, Z., & Zhang, N. (2026) · Computers in Industry · doi:10.1016/j.compind.2026.104462 · Cited by 1
UniEval-RAG: A Unified End-to-End Evaluation Framework for RAG Systems
Liang, S., Gong, X., Li, F., Zhou, H., Wu, Y., & Liao, C. (2025) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1109/ciotsc67482.2025.11412981 · Cited by 1
Guainazzo, N., Delzanno, G., Ancona, D., & D’Agostino, D. (2026) · Sensors · doi:10.3390/s26051590 · Cited by 1
When Semantic Similarity Fails: Analyzing Metric Divergence in Legal RAG Evaluation
SILVA, E. D. D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21324-2_43 · Cited by 1
FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG
Dassen, M., Kotula, R., Murray, K., Yates, A., Lawrie, D., Kayi, E. S., Mayfield, J., & Duh, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_18 · Cited by 7
Sugimoto, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/406042 · Cited by 1
The Development of Information Retrieval Model for Hadith Data from Knowledge Graph
Norhisham, N. N., Muhammad, N., Ismail, M. I. F., & Fabregas, A. C. (2026) · International Conference on Electronics, Computer and Computation · doi:10.1109/icecco67619.2026.11488814 · Cited by 1
Can We Evaluate RAGs with Synthetic Data?
Elburg, J. M. V., Putten, P. V. D., & Marx, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19102-1_11 · Cited by 6
Burillo, J. D., Neira, R. B. H., & Borrás-Gené, O. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10090299 · Cited by 1
Schnieder, M., Schimanke, F., Sert, M., Persia, F., Lyu, C., & Glesener, G. B. (2026) · International Journal of Semantic Computing · doi:10.1142/s1793351x26430038 · Cited by 1
Faridi, A. R., Masood, F., Keshvi, & Yunus, T. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70122 · Cited by 2
Tolsgaard, M. G., Grierson, L., Pusic, M. V., & Turner, L. (2026) · Medical Education · doi:10.1111/medu.70309 · Cited by 3
Kawashima, S., Shiramatsu, S., & Mizumoto, T. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3305-7_48 · Cited by 1
Avaliação do Impacto de Diferentes Padrões Arquiteturais RAG em Domínios Jurídicos
Paranhos, S. L., Tomazini, J. N., Camilo-Junior, C. G., & de Oliveira, S. S. T. (2024) · Escola Regional de Informática de Goiás (ERI-GO) · doi:10.5753/erigo.2024.4846 · Cited by 1
Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
Caruzzo, C., Yoo, D., & Kim, T. S. (2026) · arXiv · doi:10.48550/arxiv.2607.09349
Same Ranking, Different Winner: How Scoring Targets Shape LLM Memory Benchmarks
Panthi, S., & Abdelfattah, R. (2026) · arXiv · doi:10.48550/arxiv.2605.24060 · Cited by 4
Leung, E., Lumer, E., Feld, C., Huber, A., Subbiah, V. K., & Paul, K. (2026) · arXiv · doi:10.48550/arxiv.2607.08700
DeepResearchGym: A Free, Transparent, and Reproducible Sandbox for Deep Research
Coelho, J., Ning, J., He, J., Mao, K., Paladugu, A. S., Setlur, P., Jin, J., Callan, J., Magalhães, J., Martins, B., et al. (2026) · Proceedings of the 2026 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) · doi:10.1145/3805713.3820409
Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs
Belikova, J., Polev, K., Parchiev, R., & Simakov, D. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731969
Liu, Y., Yin, Z., Wang, J., & Li, K. (2025) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc65474.2025.11090835
Automated Question-Answer Generation for Evaluating Vietnamese RAG-Based Approaches
Nguyen, D.-H., & Nguyen, L. H. B. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-98164-7_21
Kim, B., & Yang, J. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-3141-7_13
RagPharma: A RAG-Based Chatbot for Medicine Leaflets with a Dual-Dataset Evaluation Framework
Navarro, L. C., Mutz, F. W., Paixão, T. M., Zanetti, G. G., Badue, C. S., Souza, A. F. D., & Oliveira-Santos, T. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5767
Reference-Aligned Retrieval-Augmented Question Answering over Heterogeneous Proprietary Documents
Choi, N., Byun, G., Chung, A., Paek, E. S., Lee, S., & Choi, J. D. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761566 · Cited by 3
TechSupportEval: An Automated Evaluation Framework for Technical Support Question Answering
Chen, B., Sun, Y., Liu, Y., Xu, L., Xie, Z., Pei, C., Han, J., Ni, F., Cai, X., Yang, C., et al. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227545
Benchmarking Retrieval-Augmented Generation: Challenging Domain-specific Datasets Generation Method
Liu, Y., Hu, L., Dong, X., & Wang, Z. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228599 · Cited by 1
Applying Large Language Models to Search
Benedetti, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01563-1_4
Enhancing Academic Search with Agentic RAG and Knowledge Graphs: A Case Study on EconBiz
Hajra, A. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10721-3_77
ChatAcadien: A RAG-LLM-Based Chatbot for Exploring Acadian Genealogy
Ghali, R., & Selouani, S. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00026 · Cited by 1
PRECISE: Reducing the bias of LLM evaluations using prediction‐powered ranking estimation
Divekar, A., & Majumder, A. (2026) · AI Magazine · doi:10.1002/aaai.70066
Liu, Y., Wang, Z., Chen, Y., He, Y., Lian, J., & Lu, H. (2025) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc67331.2025.00010
Relation-Aware Retrieval Augmented Generation for Relation Extraction
Tian, L., Bu, C., Huang, M., & Wu, X. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342441
Explainable Knowledge Access: Recursive and Rerank-Based RAG for Interpretable QA
Cosseddu, M., Giuliani, A., Manca, M. M., Pilloni, M., Podda, A. S., & Tiddia, S. G. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-15638-9_13 · Cited by 1
Nikolić, M., Stanimirović, A., & Stoimenov, L. V. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-04890-5_32
RAG-CUE: Actionable and Unified Evaluation for Retrieval-Augmented Generation Systems
Prabhakar, P. K., Shukla, N. K., & Krishnamurthy, V. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401780
Dobrovodsky, P., Eszter, B. D. K., Tafferner-Gulyás, V., & Fleiner, R. (2026) · IEEE World Symposium on Applied Machine Intelligence and Informatics (SAMI) · doi:10.1109/sami68106.2026.11420463
Talmud-IR: A Talmud-Inspired Interface for Discussing RAG Response Quality
Kusa, W., Deckers, N., Fröbe, M., Dietz, L., Platow, B., & Sanderson, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_21
Zhang, J., He, R., Zheng, X., Lin, Y., Lu, S., & Qu, Y. (2025) · International Symposium on Artificial Intelligence and Big Data (AIBDF) · doi:10.1109/aibdf67964.2025.11440704
A Graph-Based RAG System for Enhanced Information Gathering in Local Newsrooms
Pillai, R. G., Fokkens, A. S., & van Atteveldt, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25159-6_25
Persistent and Conversational Multi-Method Explainability for Trustworthy Financial AI
Makridis, G., Fatouros, G., Soldatos, J., Katsis, G., & Kyriazis, D. (2026) · International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) · doi:10.1109/dcoss-iot69657.2026.00158
Beyond the Rules: Understanding the Design Logic of Internet Standards
Bian, J., Welzl, M., & Arefyev, N. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795082
Explainable academic advising via category-routed knowledge graph retrieval and generation
Nguyen, H.-H., La, T.-T., & Ma, T.-T. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07300-3
TCK-QA: Benchmarking Knowledge Graph-Augmented LLMs for Question Answering on the Turkish Penal CodePreprint
Ay, B., & Okuyan, H. (2026) · Research Square · doi:10.21203/rs.3.rs-9804227/v1
Metadata
- Authors
- Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Venue
- Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 338-354
- OpenAlex
- W4401042808
- Cited by
- 148
- References
- 0
- Citations by year
- 2026: 84 · 2025: 48 · 2024: 16
- FWCI
- 43.61
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- retrieval-augmented generation, automated assessment, evaluation framework, large language models, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
