The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Benchmarking Retrieval-Augmented Generation for Medicine
Open accessAbstract
While large language models (LLMs) have achieved state-of-the-art performance on a wide range of medical question answering (QA) tasks, they still face challenges with hallucinations and outdated knowledge.Retrievalaugmented generation (RAG) is a promising solution and has been widely adopted.However, a RAG system can involve multiple flexible components, and there is a lack of best practices regarding the optimal RAG setting for various medical purposes.To systematically evaluate such systems, we propose the Medical Information Retrieval-Augmented Generation Evaluation (MIRAGE), a first-of-its-kind benchmark including 7,663 questions from five medical QA datasets.Using MIRAGE, we conducted large-scale experiments with over 1.8 trillion prompt tokens on 41 combinations of different corpora, retrievers, and backbone LLMs through the MEDRAG toolkit introduced in this work.Overall, MEDRAG improves the accuracy of six different LLMs by up to 18% over chain-of-thought prompting, elevating the performance of GPT-3.5 and Mixtral to GPT-4level.Our results show that the combination of various medical corpora and retrievers achieves the best performance.In addition, we discovered a log-linear scaling property and the "lostin-the-middle" effects in medical RAG.We believe our comprehensive evaluations can serve as practical guidelines for implementing RAG systems for medicine 1 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Recently, LLMs have shown the ability to encode clinical knowledge7,8, retrieve relevant medical texts9, and perform accurate single-turn medical question-answering3,10–12.”
From AgentClinic: a multimodal benchmark for tool-using clinical AI agents (2026) · cites this paper as [9]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All analyses were performed on publicly available, de-identified benchmark datasets—MIRAGE [23], PubMedQA [24], and the medical licensing examination questions of Katz et al. [25]—used in accordance with their respective licenses and terms of use.”
From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [23]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similarly, without context on the MIRAGE datset, MedGemma 27B with extended reasoning (PCCR-1024) reached 80.1% which is higher than the 73.4% accuracy reported for GPT-4 without context and matched the performance of GPT-4 with access to extensive context [23].”
From The economics of accuracy for medical reasoning with large language models (2026) · cites this paper as [23]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This pattern appeared in approximately 40% of cases, underscoring retrieval challenges in medical RAG systems where the complexity of clinical terminology limits the effectiveness of standard mechanisms [57,58].”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [57]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Such errors appeared in approximately 30% of cases, underscoring integration challenges in RAG systems where connecting retrieved facts to query-specific conclusions remain a persistent limitation [57].”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [57]
Experimental — accuracy not yet measured
In the citing paper
“The quality of RAG completion also depends on the retrieval results, such as the density or positions of query-relevant information14,22,24–26.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 45
- Asma Ben Abacha, Chaitanya Shivade, and Dina Demner-Fushman. 2019. Overview of the mediqa 2019 shared task on textual inference, question en- tailment and question answering. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 370-379.
- Waleed Ammar, Dirk Groeneveld, Chandra Bhagavat- ula, Iz Beltagy, Miles Crawford, Doug Downey, Ja- son Dunkelberger, Ahmed Elgohary, Sergey Feld- man, Vu Ha, et al. 2018. Construction of the liter- ature graph in semantic scholar. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers), pages 84-91.
- Sofia J Athenikos and Hyoil Han. 2010. Biomedical question answering: A survey. Computer methods and programs in biomedicine, 99(1):1-24.
- Asma Ben Abacha and Dina Demner-Fushman. 2019. A question-entailment approach to question answering. BMC bioinformatics, 20(1):1-23.
- Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from tril- lions of tokens. In International conference on ma- chine learning, pages 2206-2240. PMLR.
- Qingyu Chen, Jingcheng Du, Yan Hu, Vipina Kuttichi Keloth, Xueqing Peng, Kalpana Raja, Rui Zhang, Zhiyong Lu, and Hua Xu. 2023a. Large language models in biomedical natural language processing: benchmarks, baselines, and recommendations. arXiv preprint arXiv:2305.16326. · arXiv
- Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel S Weld. 2020. Specter: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2270-2282.
- Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in informa- tion retrieval, pages 758-759.
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
- Alexander R Fabbri, Wojciech Kryściński, Bryan Mc- Cann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021. Summeval: Re-evaluating summariza- tion evaluation. Transactions of the Association for Computational Linguistics, 9:391-409.
- Nicolas Fiorini, Robert Leaman, David J Lipman, and Zhiyong Lu. 2018. How user intelligence is improv- ing pubmed. Nature biotechnology, 36(10):937-945.
- Giacomo Frisoni, Miki Mizutani, Gianluca Moro, and Lorenzo Valgimigli. 2022. Bioreader: a retrieval- enhanced text-to-text transformer for biomedical lit- erature. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 5770-5793.
- Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific lan- guage model pretraining for biomedical natural lan- guage processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-23.
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language under- standing. In International Conference on Learning Representations.
- William Hersh. 2024. Search still matters: informa- tion retrieval in the era of generative ai. Journal of the American Medical Informatics Association, page ocae014.
- Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebas- tian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised dense informa- tion retrieval with contrastive learning. Transactions on Machine Learning Research.
- Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
- Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Ap- plied Sciences, 11(14):6421.
- Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Pro- ceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567-2577.
- Qiao Jin, Robert Leaman, and Zhiyong Lu. 2023b. Re- trieve, summarize, and verify: How will chatgpt im- pact information seeking from the medical literature? Journal of the American Society of Nephrology, pages 10-1681.
- Qiao Jin, Zheng Yuan, Guangzhi Xiong, Qianlan Yu, Huaiyuan Ying, Chuanqi Tan, Mosha Chen, Song- fang Huang, Xiaozhong Liu, and Sheng Yu. 2022. Biomedical question answering: a survey of ap- proaches and challenges. ACM Computing Surveys (CSUR), 55(2):1-36.
- Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781.
- Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis, and Georgios Paliouras. 2023. Bioasq- qa: A manually curated corpus for biomedical ques- tion answering. Scientific Data, 10(1):170.
- Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234-1240.
- Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng- Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2356- 2362.
- Aakanksha Naik, Sravanthi Parasa, Sergey Feldman, Lucy Wang, and Tom Hope. 2022. Literature- augmented clinical outcome prediction. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 438-453.
- OpenAI, :, Josh Achiam, Steven Adler, Sandhini Agar- wal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Alt- man, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haim- ing Bao, Mo Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christo- pher Berner, Lenny Bogdonoff, Oleg Boiko, Made- laine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowl- ing, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Is- abella Fulford, Leo Gao, Elie Georges, Christian Malte Ostendorff, Nils Rethmeier, Isabelle Augenstein, Bela Gipp, and Georg Rehm. 2022. Neighborhood contrastive learning for scientific document represen- tations with citation embeddings. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 11670-11688.
- Ankit Pal, Logesh Kumar Umapathi, and Malaikan- nan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical do- main question answering. In Conference on Health, Inference, and Learning, pages 248-260.
- PMLR.
- François Remy, Kris Demuynck, and Thomas De- meester. 2022. Biolord: Learning ontological repre- sentations from definitions for biomedical concepts and their textual descriptions. In Findings of the As- sociation for Computational Linguistics: EMNLP 2022, pages 1454-1465.
- Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and be- yond. Foundations and Trends® in Information Re- trieval, 3(4):333-389.
- Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mah- davi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023a. Large language models encode clinical knowledge. Nature, 620(7972):172-180.
- Sarvesh Soni and Kirk Roberts. 2020. Evaluation of dataset selection for pre-training and fine-tuning transformer language models for clinical question answering. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5532- 5538.
- Nandan Thakur, Nils Reimers, Andreas Rücklé, Ab- hishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Con- ference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
- Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288. · arXiv
- Daniel Truhn, Jorge S Reis-Filho, and Jakob Nikolas Kather. 2023. Large language models should be used as scientific reasoning engines, not knowledge databases. Nature medicine, 29(12):2983-2984.
- George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopou- los, et al. 2015. An overview of the bioasq large-scale biomedical semantic indexing and question answer- ing competition. BMC bioinformatics, 16(1):1-28.
- David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534-7550.
- Lucy Lu Wang, Julia Otmakhova, Jay DeYoung, Thinh Hung Truong, Bailey Kuehl, Erin Bransom, and Byron C Wallace. 2023a. Automated metrics for medical multi-document summarization disagree with human evaluations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9871- 9889.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837.
- Guillaume Wenzek, Marie-Anne Lachaux, Alexis Con- neau, Vishrav Chaudhary, Francisco Guzmán, Ar- mand Joulin, and Édouard Grave. 2020. Ccnet: Ex- tracting high quality monolingual datasets from web crawl data. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4003- 4012.
- Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Pmc-llama: Further fine- tuning llama on medical papers. arXiv preprint arXiv:2304.14454. · arXiv
- Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate nearest neigh- bor negative contrastive learning for dense text re- trieval. In International Conference on Learning Representations.
- Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D Manning, Percy S Liang, and Jure Leskovec. 2022. Deep bidirectional language-knowledge graph pretraining. Advances in Neural Information Processing Systems, 35:37309- 37323.
- Pierre Zweigenbaum. 2003. Question answering in biomedicine. In Proceedings Workshop on Natural Language Processing for Question Answering, EACL, volume 2005, pages 1-4. Citeseer.
Cited by
Sorted by citations
Items 101–200 of 314
Lin, Q., Xu, B., Hu, G., Li, Z., Hao, Z., Zhang, K., & Cai, R. (2025) · Applied Intelligence · doi:10.1007/s10489-025-07044-6 · Cited by 12
Frutuoso, J. (2025) · Cureus · doi:10.7759/cureus.92571 · Cited by 4
Few-shot phase retrieval algorithm based on lightweight networks
Su, X., Wei, X., Wu, Q., Ma, L., & Cai, C. (2025) · Physica Scripta · doi:10.1088/1402-4896/ae141b · Cited by 4
Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Tree
Peng, Q., Cui, J., Xie, J., Cai, Y., & Li, Q. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755418 · Cited by 9
Hong, G.-H., Ouyang, T., Zhao, K., Zhou, Z., & Chen, X. (2025) · IEEE Real-Time Systems Symposium · doi:10.1109/rtss66672.2025.00022 · Cited by 5
Retrieval-Augmented Generation for Medical Decision-Making in Emergency Care
Noll, R., Windschmitt, J., Hofmann, E., Bergmann, N., & Schaaf, J. (2025) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc58623.2025.11253463 · Cited by 4
Li, P., Patel, A., Vallamchetla, S. K., Heninger, H., Contractor, H., Tao, C., & Cheung, J. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag056 · Cited by 4
Yu, Y. Q., Li, L., & Li, Y. (2025) · medRxiv · doi:10.1101/2025.10.17.25338266 · Cited by 5
Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework
Aljohani, B., & Alsanoosy, T. (2026) · Information · doi:10.3390/info17020133 · Cited by 3
Embracing Foundation Models for Advancing Scientific Discovery
Guo, S., Shariatmadari, A. H., Xiong, G., & Zhang, A. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825618 · Cited by 5
Self-citationIaraMed: A Women’s Healthcare Chatbot for Portuguese Speakers
Färber, F. B., Dollis, J. S., Ribeiro, P. S. F. B., Brito, I. A., Sousa, R. T., & Filho, A. R. G. (2025) · doi:10.5753/sbcas.2025.7897 · Cited by 4
Comparação de Modelos de Embeddings e LLMs para Geração Aumentada por Recuperação em Português
Medeiros, L. S. F., & Oliveira, H. (2025) · Anais do Seminário Integrado de Software e Hardware (SEMISH) · doi:10.5753/semish.2025.9027 · Cited by 3
Hybrid-NL2SVA: Integrating RAG and Finetuning for LLM-based NL2SVA
Xiao, W., Ekberg, D., Garg, S., & Karri, R. (2025) · ACM/IEEE Workshop on Machine Learning for CAD (MLCAD) · doi:10.1109/mlcad65511.2025.11189208 · Cited by 3
Rast, J., Wiegand, S., Biermann, J., Wiegand, A., & Marschner, F. (2025) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-025-06400-w · Cited by 3
Wang, S., Shen, L. D., Tian, Z., Lan, T., Xu, Z. J., & Li, Z. (2026) · Annals of the American Association of Geographers · doi:10.1080/24694452.2026.2617563 · Cited by 3
Han, J., Burgess, P., & Shareghi, E. (2026) · Artificial Intelligence and Law · doi:10.1007/s10506-026-09506-9 · Cited by 9
Retrieval-Augmented Representation for Semi-Supervised Medical Image Segmentation
Luo, H., Zeng, Q., Zhang, Y., & Xia, Y. (2026) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2026.3712008 · Cited by 3
Tutorial: guidance on the use of large language models for medical research
Jin, Q., Wan, N., Leaman, R., Tian, S., Wang, Z., Yang, Y., Wang, Z., Xiong, G., Lai, P.-T., Zhu, Q. L., et al. (2026) · Nature Protocols · doi:10.1038/s41596-026-01408-z · Cited by 15
Self-citationA RAG Approach for Multi-Modal Open-ended Lifelog Question-Answering
Tran, Q.-L., Pham, N. N. D., Truong, Q. T., Nguyễn, M. H., Le, H. C., Vu, D. K., Nguyen, V. M. T., Nguyễn, V. K., Nguyen, L. P. N. L., Le, T., et al. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733263 · Cited by 3
Survey of Interactive Document Query Systems Using RAG and LangChain
Sambare, G. B., Ambala, S., Kadam, G., & Sonawane, V. (2025) · International Conference on Computing Technologies (ICOCT) · doi:10.1109/icoct64433.2025.11118753 · Cited by 4
Developing a GraphRAG-enabled local-LLM for Gestational Diabetes MellitusPreprint
Evangelista, E. D. L., Ruba, F., Bukhari, S. M. S., Nazir, A., & Sharma, R. S. (2025) · medRxiv · doi:10.1101/2025.04.28.25326568 · Cited by 3
Mao, B., Prasadha, M. K., Xie, Z., He, J., Ghebranious, M., Xu, H., Zhi, D., & Rasmy, L. (2026) · medRxiv · doi:10.64898/2026.04.24.26351503 · Cited by 3
Large language models in medicine: current ethical challenges
Kostrov, S. А., & Potapov, M. (2025) · Медицинская этика · doi:10.24075/medet.2025.008 · Cited by 2
Artificial intelligence in wearable biosensing: Enhancing data analysis and decision-making
Ding, Z., Fang, W., Zhang, J., Fang, C., & Sun, Y. (2025) · Progress in molecular biology and translational science · doi:10.1016/bs.pmbts.2025.06.012 · Cited by 3
Bridging guidelines and large language models: a case for retrieval-augmented generation
Alexandrou, M., Mahtani, A. U., & Brilakis, E. S. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf546 · Cited by 2
Jiang-Kells, J., Brandreth, J., Zhu, L., Ross, J., Jani, Y. H., Costanza, E., Amran, M., Kraljević, Ž., Bai, X., Cresta, R., et al. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03195-1 · Cited by 3
Khan, T., Toufiq, M., Yurieva, M., Indrawattana, N., Jittmittraphap, A., Kosoltanapiwat, N., Pumirat, P., Sukphopetch, P., Vanaporn, M., Palucka, K., et al. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf541 · Cited by 2
Zhang, Z., Hao, B., Li, Y., & Yin, H. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3607819 · Cited by 2
Enhancing Epidemiological Insights with RAG for SIREVA-SUS Reports
Freitas, C., Rabonato, R. T., & Berton, L. (2025) · doi:10.5753/eniac.2025.11796 · Cited by 2
RADDI: A Retrieval Augmented Framework for Drug-Drug Interaction Prediction
Zhang, H., Wang, Y., Gao, X., & Xiong, Y. (2026) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020059 · Cited by 2
Bai, T., Jiang, K., Yu, Y., Qie, S., Wang, C., Wang, B., & Zhang, W. (2026) · Future Internet · doi:10.3390/fi18020095 · Cited by 2
Cheema, M. N., Nazir, A., Harmanci, A. O., Harmanci, A. O., Cheema, Y., Masood, S., & Khokhar, F. A. (2026) · The Visual Computer · doi:10.1007/s00371-026-04385-2 · Cited by 2
Contradictions in Context: Challenges for Retrieval-Augmented Generation in Healthcare
Javadi, S., Mirabi, S., Gangar, M., & Ofoghi, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_3 · Cited by 3
MedSegAgent: A Universal and Scalable Multi-Agent System for Instructive Medical Image Segmentation
Huang, Z., Wang, H., Ye, J., Ji, Y., Hu, X., Liu, L., Yang, Z., Li, W., Hu, M., Su, Y., et al. (2026) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2026.3677444 · Cited by 2
Med-R 2 : Crafting Trustworthy LLM Physicians via Retrieval and Reasoning of Evidence-Based Medicine
Lu, K., Zheng, L., Pan, D., Zhang, S., Dong, G., Leng, H., Cui, B., wu, Z., & Zhang, W. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792076 · Cited by 21
Yang, W., Xue, H., Peng, Q., Hu, H., Huang, Q., & Zhang, T. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462104 · Cited by 2
Huang, Y., Gao, T., Zhang, J., Liu, X., & Wang, G. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822725 · Cited by 3
Document Embeddings Enhance Biomedical Retrieval-Augmented Generation
Kong, Y., Yang, Z., Luo, L., Ding, Z., Wang, L., Liu, W., Zhang, Y., Xu, B., Wang, J., Sun, Y., et al. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822781 · Cited by 2
Semantic Chunking and Chain-Of-Thought Reasoning for Rag-Based Document Processing
Ateş, Y., Sayar, A., Bozlar, İ. U., Ertuğrul, S., & Arslan, Ş. S. (2025) · Machine learning for signal processing ... · doi:10.1109/mlsp62443.2025.11204203 · Cited by 2
Knoll: Creating a Knowledge Ecosystem for Large Language Models
Zhao, D., Yang, D., & Bernstein, M. S. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747711 · Cited by 7
Optimizing Multimodal Algorithms for Complex Medical Data Analysis
Mounassif, F., Balouki, Y., Rifay, Y., & Lmati, I. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-95326-2_22 · Cited by 1
REANIMATOR: Reanimate Retrieval Test Collections with Extracted and Synthetic Resources
Engelmann, B., Haak, F., Schaer, P., Abdoust, M. E., Netze, L., & Bittkowski, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730342 · Cited by 3
Empowering SMEs with SustainWater Bot to advance urban water sustainability
Arslan, M., Munawar, S., & Riaz, Z. (2025) · Sustainable Cities and Society · doi:10.1016/j.scs.2025.106793 · Cited by 2
BaMCo: Balanced Multimodal Contrastive Learning for Knowledge-Driven Medical VQA
Yazıcı, Z. A., & Ekenel, H. K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04981-0_8 · Cited by 1
Learning to Find Good Hash Functions for Embeddings
Claydon, B., Connor, R., & Dearle, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06069-3_27 · Cited by 1
MediMind: A Domain-Adaptive Medical LLM Leveraging Fine-Tuning and Retrieval-Augmented Generation
Hashemi, S. M. H., Kashani, H. B., & Fatemi, S. (2025) · International Conference on the Internet of Things (IoT) · doi:10.1109/iot69654.2025.11297589 · Cited by 1
Celsa, C., Giuffrè, M., Maria, G. D., Gruttadauria, S., Palazzo, U., Miraglia, R., Maruzzelli, L., Pagano, D., Cannella, R., Midiri, F., et al. (2025) · JHEP Reports · doi:10.1016/j.jhepr.2025.101715 · Cited by 1
Hierarchical agent reflection for aligning LLM reasoning with clinical diagnostic processes
Wang, X., Li, X., Zhao, D., Feng, K., Liang, L., Zhang, Z., Ding, K., Chen, H., Wan, B., & Zhang, Q. (2026) · Health Information Science and Systems · doi:10.1007/s13755-025-00410-1 · Cited by 1
Yadav, V., Gaurav, Rana, A., & Sharma, S. K. (2025) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat66372.2025.11368379 · Cited by 1
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 1
UniEval-RAG: A Unified End-to-End Evaluation Framework for RAG Systems
Liang, S., Gong, X., Li, F., Zhou, H., Wu, Y., & Liao, C. (2025) · International Conference on Computer, Internet of Things and Smart City (CIoTSC) · doi:10.1109/ciotsc67482.2025.11412981 · Cited by 1
Push and Pull : Defending against Retrieval Poisoning Attacks via Embedding Space Reshaping
He, L., Zhang, X., Liu, Q., Li, C., Liu, Z., Zhou, P., & Su, S. (2026) · ACM Transactions on Information Systems · doi:10.1145/3800948 · Cited by 2
InterFeat: a pipeline for finding interesting scientific features
Ofer, D., Michal Linial, B., & Shahaf, D. (2026) · Scientific Reports · doi:10.1038/s41598-026-43169-5 · Cited by 1
Ge, Y., Guo, Y., Das, S., & Sarker, A. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-025-00062-2 · Cited by 6
Li, J., Chang, Y., Liu, Y., Wang, Y., Ou, B., Lin, S., Tang, S.-K., & Zheng, L. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1804478 · Cited by 1
He, N., Yang, L., Hu, X., & He, Y. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1785443 · Cited by 1
Can We Evaluate RAGs with Synthetic Data?
Elburg, J. M. V., Putten, P. V. D., & Marx, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-19102-1_11 · Cited by 6
Ngartera, L., Nadarajah, S., Rodoumta, K., & Gningue, Y. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8060151 · Cited by 1
Hierarchical context enhancement for long-tail entity retrieval augmented generation
Peng, Y., & Pan, K. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1861536 · Cited by 1
PriHA: A RAG-Enhanced LLM Framework for Primary Healthcare Assistant in Hong Kong
Chan, R. W. C., Lin, S., Ma, Y.-N., Chen, H., Jiang, L., & Fan, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1926-1_26 · Cited by 1
Schnieder, M., Schimanke, F., Sert, M., Persia, F., Lyu, C., & Glesener, G. B. (2026) · International Journal of Semantic Computing · doi:10.1142/s1793351x26430038 · Cited by 1
M3: Conversational LLMs simplify secure clinical data access, understanding, and analysis
Attrach, R. A., Moreira, P., Fani, R., Umeton, R., Fiske, A., & Celi, L. A. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001671 · Cited by 9
Zhao, J. W., Li, L., Ning, W., Hao, J., Fei, Y., & Huang, J. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822677 · Cited by 1
Ward, R., Dalal, D., Buitelaar, P., & Breslin, J. G. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707748 · Cited by 1
Instruções de Uso de Medicamentos Suportadas por RAG em Grandes Modelos de Linguagem
dos Reis, D. N., Reis, Z. S. N., & da Rocha, L. C. D. (2025) · Revista Eletrônica de Iniciação Científica em Computação · doi:10.5753/reic.2025.6039 · Cited by 1
KGRAG-Ex: Explainable Retrieval-Augmented Generation with Knowledge Graph-Based Perturbations
Balanos, G., Chasanis, E., Skianis, K., & Pitoura, E. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5009-8_3 · Cited by 1
Sekar, T., Kushal, Shankar, S., Mohammed, S., & Fiaidhi, J. (2025) · medRxiv · doi:10.1101/2025.05.03.25325604 · Cited by 2
InterFeat: An Automated Pipeline for Finding Interesting Hypotheses in Structured Biomedical DataPreprint
Ofer, D., Michal Linial, B., & Shahaf, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5342565 · Cited by 1
Yu, J., Tang, Y., Feng, K., Liang, L., Zhang, Q., Ding, K., & Chen, H. (2026) · Scientific Data · doi:10.1038/s41597-026-06594-9 · Cited by 6
The economics of accuracy for medical reasoning with large language models
Bhattacharyya, K., & Kamabattula, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001182 · Cited by 1
SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning
Huang, Y., Chen, R., & Cheng, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.917
Mmrag: Multimodal Medical Retrieval Augmented Generation SystemPreprint
Phani, S., Abdul, A., Kuo, C., Chen, J., & Dash, J. K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5021734
Reliable Typical Case Diagnosis via Optimized Retrieval-Augmented Generation Techniques
Zhang, K., Wang, B., Yuan, C., Feng, C., & Shi, G. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4298-4_19
Sekar, T., Kushal, K., Shankar, S., Mohammed, S., & Fiaidhi, J. (2025) · doi:10.36227/techrxiv.174494685.55828887/v1
Park, H., Rees, M., Kruger, N., Fuse, K., Castro, V. M., Gainer, V. S., Wattanasin, N., Benoit, B., Wagholikar, K. B., & Murphy, S. N. (2025) · medRxiv · doi:10.1101/2025.04.29.25326696 · Cited by 1
Ma, W., Fong, G., Lai, J., Wu, H., Hue, S. P. Y., Ying, J., The Hong Kong Genome Project, Chu, A. T.-W., & Chung, B. H. Y. (2025) · medRxiv · doi:10.1101/2025.06.03.25328923
Murali, S., Kamath, S. S., & Sugumaran, V. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97141-9_16
Artificial Intelligence and the Application of Augmented Reality Techniques as Disruptive Innovation
Mergh, C., Morais, F. L. D., & de Magalhães, J. L. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-4030-1.ch013
Zhang, B., Wang, S., Jiang, Y., Sui, D., Tu, Z., & Chu, D. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729898
CardioTRAP: Design of a Retrieval Augmented System (RAG) for Clinical Data in Cardiology
Guzzi, P. H., Defilippo, A., Procopio, N., Trapuzzano, A., Canino, G., Vizza, P., Sorrentino, S., Indolfi, C., & Veltri, P. (2025) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi64645.2025.00053
Bridging Knowledge Gaps: Fine-Tuned RAG Frameworks for Biomedical Evidence-Based Question Answering
Wang, X., You, Z.-H., Li, Y., Yan, Y.-H., & Li, Y.-C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0014-7_7
Sonawane, V., Sambare, G. B., Ambala, S., & Kadam, G. (2025) · International Conference on Computing Technologies (ICOCT) · doi:10.1109/icoct64433.2025.11118878 · Cited by 1
Hybrid Memory-Retrieval Model: Enhancing Trust in Medical ChatbotsPreprint
Singh, S. (2025) · Qeios · doi:10.32388/r6ojq5
Refinement-Based Graph Construction for Search in Low-Memory Systems
Foster, C., Magdaleno-Gatica, S., & Kimia, B. B. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06069-3_38
Aktar, M., Jindal, G., Bosshart, S. L., Stebner, A., Paiva, P., Bento, M., Ospel, J. M., & Souza, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07502-4_2
Ward, R., Dalal, D., Buitelaar, P., & Breslin, J. G. (2025) · ACM SIGAPP Applied Computing Review · doi:10.1145/3770865.3770867 · Cited by 1
Einsatz von Foundation-Modellen entlang der Patientenreise
Antweiler, D., Stefan Rüping, A., & Wrobel, S. (2025) · Springer eBooks · doi:10.1007/978-3-658-48573-3_40
Qi, B., & Ruan, H. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.24.684425
Enhancing Medical Question-Answering Chatbot with RAG-Based Retrieval and Summarization
Chen, O. T.-C., & Chang, C.-H. (2025) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan66881.2025.11207819
Measuring Semantic Coherence of RAG-Generated Abstracts Through Complex Network Metrics
Gana, B., Palma, W., Lucay, F. A., Missana, C., Abarza, C., & Allende‐Cid, H. (2025) · Mathematics · doi:10.3390/math13213472
RagPharma: A RAG-Based Chatbot for Medicine Leaflets with a Dual-Dataset Evaluation Framework
Navarro, L. C., Mutz, F. W., Paixão, T. M., Zanetti, G. G., Badue, C. S., Souza, A. F. D., & Oliveira-Santos, T. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5767
Benchmarking Retrieval-Augmented Generation: Challenging Domain-specific Datasets Generation Method
Liu, Y., Hu, L., Dong, X., & Wang, Z. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228599 · Cited by 1
An Enhanced Variant-Aware Deep Learning Model for Individual Gene Expression PredictionPreprint
Zhao, X., & Su, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.12.688070
Bootstrapping Post-Training for LLM Translator
Gao, W., Li, Y., Min, Q., & Zhang, Y. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3624962
Chatzimina, M. E., Tachos, N. S., Fotiadis, D. I., & Tsiknakis, M. (2025) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi67747.2025.11269561
Multi-Agent LLM Reasoning for Clinical Procedure Sequencing from High-Granularity EHR Data
Zhong, Y., Shi, W., Tamo, J. B., Nnamdi, M. C., Yuan, Y., & Wang, M. D. (2025) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3765612.3767238 · Cited by 1
RAG-NoiseLaw: A Retrieval-Augmented Framework for Noise Regulation Question
Gu, T., Luo, J., Jiang, T., Song, W., Song, X., & Quan, Y. (2025) · doi:10.1145/3761668.3761679
Munz, L. P. M., Neef, C., Preusser, I., & Richert, A. (2025) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3765766.3765779 · Cited by 1
Geng, Y., Bi, X., & De, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.5907882
Azhar, M. W., Khan, A., Nuannimnoi, S., Rijal, H., Lai, C.-H., & Huang, C. Y. (2025) · International Conference on Artificial Intelligence for Medicine, Health and Care (AIxMHC) · doi:10.1109/aixmhc65380.2025.00033
Metadata
- Authors
- Xiong, G., Jin, Q., Lu, Z., & Zhang, A.
- Affiliations
- National Institutes of Health (US)
- Venue
- Findings of the Association for Computational Linguistics ACL 2024
- Published
- 2024
- Pages
- 6233-6251
- OpenAlex
- W4402670290
- Cited by
- 310
- References
- 0
- Citations by year
- 2026: 142 · 2025: 144 · 2024: 24
- FWCI
- 91.48
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2333740)
- National Institutes of Health (1R01LM014012)
- U.S. National Library of Medicine
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- retrieval-augmented generation, medical question answering, benchmark, hallucinations, large language models, medical corpora, Retriever, chain-of-thought prompting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
