Evaluation of Retrieval-Augmented Generation: A Survey
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Retrieval-Augmented Generation (RAG) has recently gained traction in natural language processing. Numerous studies and real-world applications are leveraging its ability to enhance generative models through external information retrieval. Evaluating these RAG systems, however, poses unique challenges due to their hybrid structure and reliance on dynamic knowledge sources. To better understand these challenges, we conduct A Unified Evaluation Process of RAG (Auepora) and aim to provide a comprehensive overview of the evaluation and benchmarks of RAG systems. Specifically, we examine and compare several quantifiable metrics of the Retrieval and Generation components, such as relevance, accuracy, and faithfulness, within the current RAG benchmarks, encompassing the possible output and ground truth pairs. We then analyze the various datasets and metrics, discuss the limitations of current benchmarks, and suggest potential directions to advance the field of RAG benchmarks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453 · cited 1×
“In the inferencing step, Large Language Models (LLMs) interpret the prompted input to generate accurate and in-depth responses that align with the query’s intent and integrate the extracted information [30,8].”
BackgroundChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254 · cited 1×
“Searches in a dynamic database may lead to misleading results or contradicting the facts.”
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140 · cited 1×
“With advanced fusion structure for queries and documents [24] and the more efficient implementation of K-Nearest Neigh-bors (KNN) [42], Approximate Nearest Neighbor (ANN) [11,22] search techniques, dense retrieval methods have become practical for large-scale use.”
MethodRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1×
“Retrieval-Augmented Generation (RAG) [29] represents a significant advancement in natural language processing by enhancing the performance of generative models through integrating information retrieval techniques.”
BackgroundHuang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516 · cited 1×
“By retrieving relevant information from external sources, RAG significantly reduces the incidence of hallucinations [20] or factually incorrect generative outputs, thereby improving the content’s reliability and richness.”
BackgroundNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 3×
“The approach of employing LLMs as evaluative judges [58] is a versatile and automatic method for quality assessment, catering to instances where traditional ground truths may be elusive [13].”
BackgroundMethodHotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 2×
“Several benchmarks draw upon the part of KILT (Knowledge Intensive Language Tasks) benchmark [37] (Natural Questions [25], HotpotQA [51], and FEVER [44])and other established datasets such as SuperGLUE [46] (Mul-tiRC [9], and ReCoRD [54]) [41].”
BackgroundFEVER: a Large-scale Dataset for Fact Extraction and VERification
Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1074 · Cited by 1,343 · cited 2×
“Several benchmarks draw upon the part of KILT (Knowledge Intensive Language Tasks) benchmark [37] (Natural Questions [25], HotpotQA [51], and FEVER [44])and other established datasets such as SuperGLUE [46] (Mul-tiRC [9], and ReCoRD [54]) [41].”
BackgroundEmergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158 · cited 1×
“The generation component utilizes the retrieved content to formulate coherent and contextually relevant responses with the prompting and inferencing phases.”
MethodSuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
ERASER: A Benchmark to Evaluate Rationalized NLP Models
DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., & Wallace, B. C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.408 · Cited by 561
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556 · cited 1×
“Research showcases various strategic prompting tactics such as the Chain of Thought (CoT) [48], Tree of Thought (ToT) [3], and Self-Note [26], each significantly shaping the model’s output.”
BackgroundGraph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.09687 · Cited by 556
Benchmarking Large Language Models in Retrieval-Augmented Generation
Chen, J., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2309.01431 · Cited by 455 · cited 10×
“The searching utilizes these indexes to fetch relevant documents on the user’s query, often incorporating the optional rerankers [4,34,5,43] to refine the ranking of the retrieved documents.”
BackgroundMethodInfluentialKILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.200 · Cited by 357
Benchmarking Retrieval-Augmented Generation for Medicine
Xiong, G., Jin, Q., Lu, Z., & Zhang, A. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.372 · Cited by 310
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240 · cited 1×
“To clarify the elements further, we conducted this survey on RAG evaluation to address the current gaps in the area, which differs from the prior RAG surveys [57,14,21] that predominantly collected specific RAG methods or data.”
MethodThe power of noise: Redefining retrieval for rag systems
Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S., Campagnano, C., Maarek, Y., Tonellotto, N., & Silvestri, F. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657834 · Cited by 236
RAGAs: Automated Evaluation of Retrieval Augmented Generation
Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2023) · arXiv · doi:10.48550/arxiv.2309.15217 · Cited by 79
RealTime QA: What's the Answer Right Now?
Kasai, J., Sakaguchi, K., Takahashi, Y., Bras, R. L., Asai, A., Yu, X., Radev, D., Smith, N. A., Choi, Y., & Inui, K. (2022) · arXiv · doi:10.48550/arxiv.2207.13332 · Cited by 52
RealTime QA: What's the Answer Right Now?
Kasai, J., Sakaguchi, K., Takahashi, Y., Bras, R. L., Asai, A., Yu, X., Radev, D., Smith, N., Choi, Y., & Inui, K. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2130 · Cited by 52
FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation
Hofstätter, S., Chen, J., Raman, K., & Zamani, H. (2022) · arXiv · doi:10.48550/arxiv.2209.14290 · Cited by 51
FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation
Hofstätter, S., Chen, J., Raman, K., & Zamani, H. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591687 · Cited by 51
Liang, X., Song, S., Niu, S., Li, Z., Xiong, F., Tang, B., Wang, Y., He, D., Cheng, P., Wang, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.15296 · Cited by 43
Learning to Reason and Memorize with Self-Notes
Lanchantin, J., Toshniwal, S., Weston, J., Szlam, A., & Sukhbaatar, S. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-0520 · Cited by 9
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M. (2023) · arXiv · doi:10.48550/arxiv.2311.09476 · Cited by 7
Researchy Questions: A Dataset of Multi-Perspective, Decompositional Questions for LLM Web Agents
Rosset, C., Chung, H.-L., Qin, G., Chau, E. C., Feng, Z., Awadallah, A., Neville, J., & Rao, N. (2024) · arXiv · doi:10.48550/arxiv.2402.17896 · Cited by 6
Research works printed but not linked: 37
- Balaguer, A., et al.: RAG vs fine-tuning: pipelines, tradeoffs, and a case study on agriculture. Technical report (2024). arXiv:2401.08406 [cs] type: article · arXiv
- Barnett, S., Kurniawan, S., Thudumu, S., Brannelly, Z., Abdelrazek, M.: Seven failure points when engineering a retrieval augmented generation system (2024). https://doi.org/10.48550/ARXIV.2401.05856 · DOI
- Deng, Y., Zhang, W., Chen, Z., Gu, Q.: Rephrase and respond: let large language models ask better questions for themselves (2023). https://doi.org/10.48550/ARXIV.2311.04205 · DOI · Paper page
- Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., Weston, J.: Wizard of Wikipedia: knowledge-powered conversational agents. In: Proceedings of the International Conference on Learning Representations (ICLR) (2019)
- Fisch, A., Talmor, A., Jia, R., Seo, M., Choi, E., Chen, D.: MRQA 2019 shared task: evaluating generalization in reading comprehension. In: Fisch, A., Talmor, A., Jia, R., Seo, M., Choi, E., Chen, D. (eds.) Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pp. 1–13. Association for Computational Linguistics, Hong Kong, China (2019). https://doi.org/10.18653/v1/D19-5801. https://aclanthology.org/D19-5801 · DOI · Paper page
- Gienapp, L., et al.: Evaluating generative ad hoc information retrieval. Technical report (2023). arXiv:2311.04694 [cs] type: article · arXiv
- Gottschalk, S., Demidova, E.: Eventkg: a multilingual event-centric temporal knowledge graph (2018). https://doi.org/10.48550/ARXIV.1804.04526 · arXiv · doi:10.48550/arxiv.1804.04526
- Huang, J., Shao, H., Chang, K.C.C., Xiong, J., Hwu, W.: Understanding jargon: combining extraction and generation for definition modeling. In: Proceedings of EMNLP (2022) · DOI
- Huang, Y., Huang, J.: A survey on retrieval-augmented text generation for large language models (2024). https://doi.org/10.48550/ARXIV.2404.10981 · DOI
- Kamalloo, E., Thakur, N., Lassance, C., Ma, X., Yang, J.H., Lin, J.: Resources for brewing beir: reproducible reference models and an official leaderboard (2023) · DOI
- Khattab, O., Zaharia, M.: Colbert: efficient and effective passage search via contextualized late interaction over BERT (2020). https://doi.org/10.48550/ARXIV.2004.12832 · arXiv · doi:10.48550/arxiv.2004.12832 · Paper page
- Lahitani, A.R., Permanasari, A.E., Setiawan, N.A.: Cosine similarity to determine similarity measure: study case in online essay assessment. In: 2016 4th International Conference on Cyber and IT Service Management, pp. 1–6 (2016). https://doi.org/10.1109/CITSM.2016.7577578 · DOI · Paper page
- Lewis, P., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS 2020, pp. 9459–9474. Curran Associates Inc., Red Hook (2020)
- Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81. Association for Computational Linguistics, Barcelona, Spain (2004). https://aclanthology.org/W04-1013
- Liu, Y., et al.: Recall: a benchmark for LLMs robustness against external counterfactual knowledge (2023). https://doi.org/10.48550/ARXIV.2311.08147 · DOI
- Lyu, Y., et al.: Crud-rag: a comprehensive Chinese benchmark for retrieval-augmented generation of large language models (2024). https://doi.org/10.48550/ARXIV.2401.17043 · DOI
- OpenAI, Achiam, J., et al.: GPT-4 Technical Report (2023). https://doi.org/10.48550/ARXIV.2303.08774 · DOI · Paper page
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
- Ramos, J., et al.: Using TF-IDF to determine word relevance in document queries. In: Proceedings of the First Instructional Conference on Machine Learning, vol. 242, pp. 29–48. Citeseer (2003)
- Robertson, S., Zaragoza, H., et al.: The probabilistic relevance framework: BM25 and beyond. Found. Trends® Inf. Retrieval 3(4), 333–389 (2009) · DOI · Paper page
- Sai, A.B., Mohankumar, A.K., Khapra, M.M.: A survey of evaluation metrics used for NLG systems. ACM Comput. Surv. (CSUR) 55(2), 1–39 (2022) · DOI · Paper page
- Shahabi, C., Kolahdouzan, M.R., Sharifzadeh, M.: A road network embedding technique for k-nearest neighbor search in moving object databases. In: Proceedings of the 10th ACM International Symposium on Advances in Geographic Information Systems, pp. 94–100 (2002) · doi:10.1145/585147.585167
- Tang, Y., Yang, Y.: Multihop-rag: benchmarking retrieval-augmented generation for multi-hop queries (2024). https://doi.org/10.48550/ARXIV.2401.15391 · DOI · Paper page
- Vaswani, A., et al.: Attention is all you need (2017). https://doi.org/10.48550/ARXIV.1706.03762 · DOI · Paper page
- Wang, S., Khramtsova, E., Zhuang, S., Zuccon, G.: Feb4rag: evaluating federated search in the context of retrieval augmented generation (2024). https://doi.org/10.48550/ARXIV.2402.11891 · DOI
- Wang, S., et al.: Domainrag: a Chinese benchmark for evaluating domain-specific retrieval-augmented generation (2024). https://doi.org/10.48550/ARXIV.2406.05654 · DOI · Paper page
- Xu, Z., et al.: Let LLMs take on the latest challenges! a Chinese dynamic question answering benchmark (2024). https://doi.org/10.48550/ARXIV.2402.19248 · DOI
- Yao, J.Y., Ning, K.P., Liu, Z.H., Ning, M.N., Yuan, L.: LLM lies: hallucinations are not bugs, but features as adversarial examples. arXiv preprint arXiv:2310.01469 (2023) · arXiv
- Yu, X., Cheng, H., Liu, X., Roth, D., Gao, J.: ReEval: automatic hallucination evaluation for retrieval-augmented large language models via transferable adversarial attacks. In: Duh, K., Gomez, H., Bethard, S. (eds.) Findings of the Association for Computational Linguistics: NAACL 2024, pp. 1333–1351. Association for Computational Linguistics, Mexico City, Mexico (2024). https://aclanthology.org/2024.findings-naacl.85 · DOI
- Zhang, K., et al.: EATN: an efficient adaptive transfer network for aspect-level sentiment analysis. IEEE Trans. Knowl. Data Eng. 35(1), 377–389 (2021)
- Zhang, K., Zhang, H., Liu, Q., Zhao, H., Zhu, H., Chen, E.: Interactive attention transfer network for cross-domain sentiment classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5773–5780 (2019) · DOI
- Zhang, K., et al.: Incorporating dynamic semantics into pre-trained language model for aspect-based sentiment analysis. arXiv preprint arXiv:2203.16369 (2022) · DOI
- Zhang, Q., et al.: A survey for efficient open domain question answering. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14447–14465. Association for Computational Linguistics, Toronto, Canada (2023). https://doi.org/10.18653/v1/2023.acl-long.808. https://aclanthology.org/2023.acl-long.808 · DOI
- Zhang, S., Liu, X., Liu, J., Gao, J., Duh, K., Van Durme, B.: Record: bridging the gap between human and machine commonsense reading comprehension (2018). https://doi.org/10.48550/ARXIV.1810.12885 · arXiv · doi:10.48550/arxiv.1810.12885 · Paper page
- Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: evaluating text generation with BERT. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020). https://openreview.net/forum?id=SkeHuCVFDr
- Zhang, Y., Khalifa, M., Logeswaran, L., Lee, M., Lee, H., Wang, L.: Merging generated and retrieved knowledge for open-domain QA. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4710–4728. Association for Computational Linguistics, Singapore (2023). https://doi.org/10.18653/v1/2023.emnlp-main.286. https://aclanthology.org/2023.emnlp-main.286 · DOI
- Zhu, F., Lei, W., Wang, C., Zheng, J., Poria, S., Chua, T.S.: Retrieving and reading: a comprehensive survey on open-domain question answering. Technical report (2021). arXiv:2101.00774 [cs] type: article · arXiv
Cited by
A survey on knowledge-oriented retrieval-augmented generation
Cheng, M., Luo, Y., Ouyang, J., Liu, Q., Liu, H., Li, L., Yu, S., Zhang, B., Cao, J., Ma, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.10677 · Cited by 140
Self-citationEvaluation of large language models for discovery of gene set function
Hu, M., Alkhairy, S., Lee, I., Pillich, R. T., Fong, D., Smith, K., Bachelder, R., Ideker, T., & Pratt, D. (2025) · Nature Methods · doi:10.1038/s41592-024-02525-x · Cited by 104
“A first major direction would be to further boost LLM accuracy and interpretability, for which recent techniques such as fine-tuning 31 and retrieval-augmented generation 62 are showing considerable promise.”
Wang, X., Huey, S. L., Sheng, R., Mehta, S., & Wang, F. (2025) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.70073 · Cited by 13
Yager, K. G. (2024) · Digital Discovery · doi:10.1039/d4dd00178h · Cited by 12
Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework
Aljohani, B., & Alsanoosy, T. (2026) · Information · doi:10.3390/info17020133 · Cited by 3
Metadata
- Authors
- Yu, H., Gan, A., Zhang, K., Tong, S., Liu, Q., & Liu, Z.
- Venue
- Communications in Computer and Information Science
- Publisher
- Springer Nature Singapore
- Published
- 2025
- Pages
- 102-120
- ISSN
- 1865-0929 (ISSN-L)
- 1865-0937
- OpenAlex
- W4406771321
- Cited by
- 173
- References
- 35
- FWCI
- 168.43
- Type
- conference paper
- Language
- en
- Open access
- green
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.