Retrieval augmented generation for large language models in healthcare: A systematic review
Open accessThe DOI matches its Crossref record; 161 of the 161 references it cites were found in a registry or an index.
Abstract
Large Language Models (LLMs) have demonstrated promising capabilities to solve complex tasks in critical sectors such as healthcare. However, LLMs are limited by their training data which is often outdated, the tendency to generate inaccurate (“hallucinated”) content and a lack of transparency in the content they generate. To address these limitations, retrieval augmented generation (RAG) grounds the responses of LLMs by exposing them to external knowledge sources. However, in the healthcare domain there is currently a lack of systematic understanding of which datasets, RAG methodologies and evaluation frameworks are available. This review aims to bridge this gap by assessing RAG-based approaches employed by LLMs in healthcare, focusing on the different steps of retrieval, augmentation and generation. Additionally, we identify the limitations, strengths and gaps in the existing literature. Our synthesis shows that 78.9% of studies used English datasets and 21.1% of the datasets are in Chinese. We find that a range of techniques are employed RAG-based LLMs in healthcare, including Naive RAG, Advanced RAG, and Modular RAG. Surprisingly, proprietary models such as GPT-3.5/4 are the most used for RAG applications in healthcare. We find that there is a lack of standardised evaluation frameworks for RAG-based applications. In addition, the majority of the studies do not assess or address ethical considerations related to RAG in healthcare. It is important to account for ethical challenges that are inherent when AI systems are implemented in the clinical setting. Lastly, we highlight the need for further research and development to ensure responsible and effective adoption of RAG in the medical domain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 PRISMA [25] workflow applied to the search, identification and selection of the studies that were included in the systematic review.](https://figures.cochat.ai/paper-figures/558117d130e94385a047aeb76b2922c7/retrieval-augmented-generation-for-large-language-models-in-healthcare-a-figure-1.jpg)



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Some general-purpose LLMs, including GPT-4-based models, can be customized through methods such as prompt engineering and in-context learning (e.g., PharmacyGPT), 10 and augmented with domain-specific external knowledge sources, enabling responses grounded in relevant databases or documents.114”
From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [114]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“Models such as BioBERT [8] and ClinicalBERT [9] have been proposed, leveraging the power of Bidirectional Encoder Representations from Transformers (BERT) [10].”
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · cited 1×
“A recent study by Jin et al. [69] proposed a framework that integrates advanced techniques such as large-scale feature extraction combined with RAG, accurate scoring of features from medical knowledge using LlamaIndex, and XGBoost [131] algorithm for prediction.”
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 1×
“All studies included met the inclusion criteria shown in Table 1.”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 1×
“Models such as BioBERT [8] and ClinicalBERT [9] have been proposed, leveraging the power of Bidirectional Encoder Representations from Transformers (BERT) [10].”
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“This issue is often referred to as "model hallucinations" [6] and methodologies for its mitigation are still an active area of research [7].”
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“To address this contextual need in medicine, Med-Palm was introduced demonstrating good performance in retrieving clinical knowledge and excelling in decision-making on several clinical tasks [11].”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083 · cited 1×
“To address this contextual need in medicine, Med-Palm was introduced demonstrating good performance in retrieving clinical knowledge and excelling in decision-making on several clinical tasks [11].”
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1×
“To address the aforementioned limitations, a novel approach called Retrieval-Augmented Generation (RAG) was proposed to expose the model to external knowledge sources [12].”
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
Alsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., & McDermott, M. (2019) · Proceedings of the 2nd Clinical Natural Language Processing Workshop · doi:10.18653/v1/w19-1909 · Cited by 1,736 · cited 1×
“Models such as BioBERT [8] and ClinicalBERT [9] have been proposed, leveraging the power of Bidirectional Encoder Representations from Transformers (BERT) [10].”
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526 · cited 1×
“This issue is often referred to as “model hallucinations” [6] and methodologies for its mitigation are still an active area of research [7].”
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876 · cited 7×
“Pre-retrieval methods aim to optimize data indexing and can be achieved using different strategies such as improving data granularity, optimizing indexing structures, adding metadata, optimizing alignment and using mixed retrieval [16].”
InfluentialChen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., & Liu, Z. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.137 · Cited by 757 · cited 1×
“Other embedding models used included BAAI general embedding (BGE) [67,89], and HuggingFace’s all-MiniLM-L6-v2 [22].”
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685 · cited 1×
“Other evaluation datasets for QA tasks include MedMCQA [34], Pub-MedQA [35] and MedQA [36].”
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.64 · Cited by 649 · cited 1×
“MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions, or products referred to in the content. retrieving relevant information from the knowledge source, and then using the relevant information to generate a response…”
Tsatsaronis, G., Balikas, G., Malakasiotis, P., Partalas, I., Zschunke, M., Alvers, M. R., Weissenborn, D., Krithara, A., Petridis, S., Polychronopoulos, D., et al. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0564-6 · Cited by 598 · cited 1×
“The Biomedical QA (BioASQ) dataset was proposed through a series of competitions as a benchmark dataset to assess systems and methodologies for large scale medical semantic indexing and QA tasks [30].”
C-Pack: Packed Resources For General Chinese Embeddings
Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657878 · Cited by 523
C-Pack: Packed Resources For General Chinese Embeddings
Xiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2023) · arXiv · doi:10.48550/arxiv.2309.07597 · Cited by 514
Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.14070 · Cited by 512 · cited 3×
“For instance, when external databases are accessed in RAG systems, there is a risk of inadvertently revealing sensitive information such as patient prescription information [58].”
Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., & Zhang, Y. (2023) · Cureus · doi:10.7759/cureus.40895 · Cited by 505 · cited 3×
“For instance, when external databases are accessed in RAG systems, there is a risk of inadvertently revealing sensitive information such as patient prescription information [58].”
Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L., & Jégou, H. (2026) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3618474 · Cited by 478
Almanac — Retrieval-Augmented Language Models for Clinical Medicine
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., et al. (2024) · NEJM AI · doi:10.1056/aioa2300068 · Cited by 424 · cited 4×
“To address this limitation, Zakka et al. [24] curated ClinicalQA as a benchmark dataset containing open-ended questions for different medical specialties, including treatment guidelines recommendations.”
InfluentialPrecise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.99 · Cited by 362 · cited 2×
“Many researchers have proposed RAG as a way to provide LLMs with up-to-date and user-specific information, not available as part of the LLM’s pre-trained knowledge (also known as "grounding the LLM) [18–24].”
Precise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2022) · arXiv · doi:10.48550/arxiv.2212.10496 · Cited by 362 · cited 2×
“Many researchers have proposed RAG as a way to provide LLMs with up-to-date and user-specific information, not available as part of the LLM’s pre-trained knowledge (also known as "grounding the LLM) [18–24].”
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.557 · Cited by 349
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · arXiv · doi:10.48550/arxiv.2212.10509 · Cited by 349 · cited 1×
“Finally, with continuous improvements LLMs will play an instrumental role in shaping the future of healthcare, driving innovation and enhancing patient care.”
Benchmarking Retrieval-Augmented Generation for Medicine
Xiong, G., Jin, Q., Lu, Z., & Zhang, A. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.372 · Cited by 310
A Full-Text Learning to Rank Dataset for Medical Information Retrieval
Boteva, V., Gholipour, D., Sokolov, A., & Riezler, S. (2016) · European Conference on Information Retrieval · doi:10.1007/978-3-319-30671-1_58 · Cited by 288
Matching patients to clinical trials with large language models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2024) · Nature Communications · doi:10.1038/s41467-024-53081-z · Cited by 287 · cited 2×
“In the realm of clinical development, advanced RAG techniques have been explored for clinical trial patient matching [78,88] and to accurately identify and report on inclusion and exclusion criteria for a clinical trial [74].”
Matching Patients to Clinical Trials with Large Language Models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2023) · arXiv · doi:10.48550/arxiv.2307.15051 · Cited by 287 · cited 2×
“In the realm of clinical development, advanced RAG techniques have been explored for clinical trial patient matching [78,88] and to accurately identify and report on inclusion and exclusion criteria for a clinical trial [74].”
Information Retrieval: Recent Advances and Beyond
Hambarde, K., & Proença, H. P. M. C. (2023) · IEEE Access · doi:10.1109/access.2023.3295776 · Cited by 284 · cited 1×
“On the other hand, information retrieval datasets are geared towards extracting and presenting relevant information to the user’s query (generally extracted from large datasets) [27].”
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.00234 · Cited by 248 · cited 1×
“MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions, or products referred to in the content. retrieving relevant information from the knowledge source, and then using the relevant information to generate a response…”
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240 · cited 1×
“Despite the growth in RAG related research, we only came across a few review studies that outline the state-of-the-art in RAG [16] and methodologies for retrievers and generators [17].”
The power of noise: Redefining retrieval for rag systems
Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S., Campagnano, C., Maarek, Y., Tonellotto, N., & Silvestri, F. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657834 · Cited by 236 · cited 1×
“In the context of this review, we discuss the RAG techniques used in healthcare by grouping them into these three categories.”
Kresevic, S., Giuffrè, M., Ajcevic, M., Accardo, A., Crocè, L. S., & Shung, D. L. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01091-y · Cited by 207
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., & Chen, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.620 · Cited by 200
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
Shao, Z., Gong, Y., Shen, Y., Huang, M., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.15294 · Cited by 200 · cited 1×
“Finally, with continuous improvements LLMs will play an instrumental role in shaping the future of healthcare, driving innovation and enhancing patient care.”
Ge, J., Sun, S., Owens, J., Galvez, V., Gologorskaya, O., Lai, J. C., Pletcher, M. J., & Lai, K. (2024) · Hepatology · doi:10.1097/hep.0000000000000834 · Cited by 178
Jin, Q., Yang, Y., Chen, Q., & Lu, Z. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae075 · Cited by 176
Miao, J., Thongprayoon, C., Suppadungsuk, S., Garcia Valencia, O. A., & Cheungpasitporn, W. (2024) · Medicina · doi:10.3390/medicina60030445 · Cited by 164 · cited 1×
“In a nutshell, these concerns can be addressed by implementing robust data privacy measures, promoting transparency and accountability, mitigating bias, emphasizing human oversight, and promoting human-autonomy.”
From local to global: a graph rag approach to query-focused summarization
Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024) · arXiv · doi:10.48550/arxiv.2404.16130 · Cited by 159 · cited 1×
“Unlike the knowledge found in document repositories, KGs provide structured and inferable information, making them more suitable for augmenting LLMs-RAG [125,126].”
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2022) · arXiv · doi:10.48550/arxiv.2203.14371 · Cited by 157 · cited 1×
“Other evaluation datasets for QA tasks include MedMCQA [43], PubMedQA[33] and MedQA[34].”
Freyer, O., Wiest, I. C., Kather, J. N., & Gilbert, S. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00124-9 · Cited by 157 · cited 1×
“Data Analysis Figure 3a presents the distribution of languages of common retrieval datasets used for RAG in medical domain.”
Alkhalaf, M., Yu, P., Yin, M., & Deng, C. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104662 · Cited by 156
The Oxford Handbook of Bioethics
Steinbock, B. (2009) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199562411.001.0001 · Cited by 152
Med-HALT: Medical Domain Hallucination Test for Large Language Models
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2023) · Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2023.conll-1.21 · Cited by 152
Med-HALT: Medical Domain Hallucination Test for Large Language Models
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15343 · Cited by 152 · cited 2×
“Finally, with continuous improvements LLMs will play an instrumental role in shaping the future of healthcare, driving innovation and enhancing patient care.”
Ke, Y. H., Jin, L., Elangovan, K., Abdullah, H. R., Liu, N., Sia, A. T. H., Soh, C. R., Tung, J. Y. M., Ong, J. C. L., Kuo, C.-F., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01519-z · Cited by 151
Rau, A., Rau, S., Zöller, D., Fink, A., Tran, H., Wilpert, C., Nattenmüller, J., Neubauer, J., Bamberg, F., Reisert, M., et al. (2023) · Radiology · doi:10.1148/radiol.230970 · Cited by 150
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)
Zeng, S., Zhang, J., He, P., Liu, Y., Xing, Y., Xu, H., Ren, J., Chang, Y., Wang, S., Yin, D., et al. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.267 · Cited by 149
Adversarial machine learning: a taxonomy and terminology of attacks and mitigations
Vassilev, A., Oprea, A., Fordyce, A., & Anderson, H. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 147
Biomedical knowledge graph-optimized prompt generation for large language models
Soman, K., Rose, P. W., Morris, J. H., Akbas, R. E., Smith, B., Peetoom, B., Villouta-Reyes, C., Cerono, G., Shi, Y., Rizk-Jackson, A., et al. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae560 · Cited by 135
Jeong, M., Sohn, J., Sung, M., & Kang, J. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae238 · Cited by 132 · cited 1×
“Other studies used custom embedding models, as in [23].”
Multimodal LLMs for Health Grounded in Individual-Specific Data
Belyaeva, A., Cosentino, J., Hormozdiari, F., Eswaran, K., Shetty, S., Corrado, G. S., Carroll, A. W., McLean, C. Y., & Furlotte, N. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47679-2_7 · Cited by 128
TREC-COVID: rationale and structure of an information retrieval shared task for COVID-19
Roberts, K., Alam, T., Bedrick, S., Demner-Fushman, D., Lo, K., Soboroff, I., Voorhees, E., Wang, L. L., & Hersh, W. R. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa091 · Cited by 123 · cited 1×
“Lastly, the compressive review includes studies that employed innovative RAG-based approaches addressing questions such as “what information to retrieve”, “when to retrieve” and “how to use the retrieved information” to ground LLMs in the medical domain.”
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., Song, J., & Zhang, T. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.585 · Cited by 117
Unlu, O., Shin, J., Mailly, C. J., Oates, M. F., Tucci, M. R., Varugheese, M., Wagholikar, K., Wang, F., Scirica, B. M., Blood, A. J., et al. (2024) · doi:10.1101/2024.02.08.24302376 · Cited by 117
Tucci, V., Saary, J., & Doyle, T. E. (2022) · Journal of Medical Artificial Intelligence · doi:10.21037/jmai-21-25 · Cited by 116
Zhao, X., Liu, S., Yang, S.-Y., & Miao, C. (2025) · Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3696410.3714782 · Cited by 116 · cited 2×
“Another key observation from our review is the language bias in the datasets used as external knowledge sources.”
Retrieval-Augmented Generation–Enabled GPT-4 for Clinical Trial Screening
Unlu, O., Shin, J., Mailly, C. J., Oates, M. F., Tucci, M. R., Varugheese, M., Wagholikar, K., Wang, F., Scirica, B. M., Blood, A. J., et al. (2024) · NEJM AI · doi:10.1056/aioa2400181 · Cited by 114 · cited 3×
“Another study used a multi-layered approach to ensure data privacy and security when using Azure OpenAI for sensitive healthcare data [74].”
Gao, Y., Li, R., Croxford, E., Caskey, J., Patterson, B. W., Churpek, M., Miller, T., Dligach, D., & Afshar, M. (2025) · JMIR AI · doi:10.2196/58670 · Cited by 114
Matsumoto, N., Moran, J., Choi, H., Hernandez, M. E., Venkatesan, M., Wang, P., & Moore, J. H. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae353 · Cited by 108 · cited 1×
“In the context of this review, we discuss the RAG techniques used in healthcare by grouping them into these three categories.”
Morris, J. H., Soman, K., Akbas, R. E., Zhou, X., Smith, B., Meng, E. C., Huang, C. C., Cerono, G., Schenk, G., Rizk-Jackson, A., et al. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad080 · Cited by 99 · cited 1×
“Soman et al. [73] proposed a context-aware prompt framework that adeptly retrieves biomedical context from the Scalable Precision Medicine Open Knowledge Engine (SPOKE) [87].”
Wu, S., Koo, M., Blum, L., Black, A., Kao, L., Fei, Z., Scalzo, F., & Kurtz, I. (2024) · NEJM AI · doi:10.1056/aidbp2300092 · Cited by 97 · cited 2×
“For example, GPT-4 and Claude 2 have consistently outperformed their open-source counterparts in these tasks [106], illustrating the strong capabilities and potential for off-the-shelf LLMs in solving complex problems in medicine.”
A benchmark for automatic medical consultation system: frameworks, tasks and datasets
Chen, W., Li, Z., Fang, H., Yao, Q., Zhong, C., Hao, J., Zhang, Q., Huang, X., Peng, J., & Wei, Z. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac817 · Cited by 96 · cited 2×
“To address these issues, Chen et al. [36] proposed an extensive medical dialogue dataset with multi-level fine-grained annotations comprising five separate tasks which include named entity recognition (NER), dialogue act classification, symptom label inference, medical report generation, and…”
MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation
Liu, W., Tang, J., Yi, C. X., Li, W., Zheng, Y., & Liang, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17120-8_35 · Cited by 94 · cited 2×
“In another study, a name entity dataset called MedDG was created from 17,000 conversations related to 12 types of common gastrointestinal diseases, collected from an online health consultation community [29].”
Proceedings of the Conference on Health, Inference, and Learning
(2021) · doi:10.1145/3450439 · Cited by 93
Wang, C., Ong, J., Wang, C., Ong, D., Cheng, R., & Ong, D. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03327-6 · Cited by 88 · cited 1×
“In addition, by incorporating up-to-date clinical data and reliable medical sources such as clinical guidelines into LLMs, the latter can offer more personalised patient advice, quicker diagnostic and treatment suggestions and significantly enhance patient outcomes [15].”
Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI
Ahmad, M., Yaramic, I., & Roy, T. D. (2023) · Preprints.org · doi:10.20944/preprints202310.1662.v1 · Cited by 84 · cited 1×
“Other researchers have proposed overcoming hal-lucination by using human-in-the-loop, algorithmic corrections and fine-tuning [159].”
Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI
Ahmad, M. A., Yaramis, I., & Roy, T. D. (2023) · arXiv · doi:10.48550/arxiv.2311.01463 · Cited by 84 · cited 1×
“Other researchers have proposed overcoming hal-lucination by using human-in-the-loop, algorithmic corrections and fine-tuning [159].”
Towards General Text Embeddings with Multi-stage Contrastive Learning
Li, Z., Zhang, X., Zhang, Y., Long, D., Xie, P., & Zhang, M. (2023) · arXiv · doi:10.48550/arxiv.2308.03281 · Cited by 79 · cited 1×
“5T gte-base-zh[118] Chunking SS FT embedding RAG improve the accuracy and reliability of LLM ∗ Accu racy (Acc); Advanced Methodologies (Adv.”
Wang, D., Liang, J., Ye, J., Li, J., Li, J., Zhang, Q., Hu, Q., Pan, C., Wang, D., Liu, Z., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/58041 · Cited by 76
Azimi, I., Qi, M., Li, W., Rahmani, A. M., & Li, Y. (2025) · Scientific Reports · doi:10.1038/s41598-024-85003-w · Cited by 76 · cited 1×
“This can be attributed to several factors: 1) the information contained in the pre-training of the model may leak in the final model answer; 2) irrelevant information may be retrieved, leading to inaccurate responses; 3) LLM generation can be unpredictable, resulting in unexpected outputs.”
The Faiss libraryPreprint
Douze, M., Guzhva, A., Deng, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.08281 · Cited by 75
Invigorating Ubuntu Ethics in AI for healthcare: Enabling equitable care
Amugongo, L. M., Bidwell, N. J., & Corrigan, C. C. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594024 · Cited by 73 · cited 1×
“Representation is crucial for fairness and equity in medical AI systems [108,109].”
Self-citationAn automated framework for assessing how well LLMs cite relevant medical references
Wu, K., Wu, E., Wei, K., Zhang, A., Casasola, A., Nguyen, T., Riantawan, S., Shi, P., Ho, D., & Zou, J. (2025) · Nature Communications · doi:10.1038/s41467-025-58551-6 · Cited by 72
Amugongo, L. M., Kriebitz, A., Boch, A., & Lütge, C. (2022) · Healthcare · doi:10.3390/healthcare11010059 · Cited by 72 · cited 1×
“Finally, with continuous improvements LLMs will play an instrumental role in shaping the future of healthcare, driving innovation and enhancing patient care.”
Self-citationDevelopment and Evaluation of a Retrieval-Augmented Large Language Model Framework for Ophthalmology
Luo, M.-J., Pang, J., Bi, S., Lai, Y., Zhao, J., Shang, Y., Cui, T., Yang, Y., Lin, Z., Zhao, L., et al. (2024) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2024.2513 · Cited by 70 · cited 1×
“Various studies have indicated that RAG reduces hallucinations [19, 54,78,81,82,146].”
Amugongo, L. M., Kriebitz, A., Boch, A., & Lütge, C. (2025) · AI and Ethics · doi:10.1007/s43681-023-00331-3 · Cited by 68 · cited 1×
“Furthermore, it is crucial for LLM developers in healthcare to proactively address ethical issues throughout the AI development life cycle [120].”
Self-citationEmpowering personalized pharmacogenomics with generative AI solutions
(2024) · doi:10.1093/jamia/ocae039 · Cited by 65
Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions
Xiong, G., Jin, Q., Wang, X., Zhang, M., Lu, Z., & Zhang, A. (2024) · Biocomputing 2025 · doi:10.1142/9789819807024_0015 · Cited by 64 · cited 1×
“Finally, with continuous improvements LLMs will play an instrumental role in shaping the future of healthcare, driving innovation and enhancing patient care.”
PRAG: Paninian Retrieval-Augmented Generation for Safety-Critical Medical Question Answering
Jin, D., Pan, E., Oufattole, N., Weng, W., Fang, H., & Peter Szolovits, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.13081 · Cited by 63 · cited 1×
“Other evaluation datasets for QA tasks include MedMCQA [43], PubMedQA[33] and MedQA[34].”
MedExpQA: Multilingual benchmarking of Large Language Models for Medical Question Answering
Alonso, I., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102938 · Cited by 62 · cited 1×
“Lastly, the compressive review includes studies that employed innovative RAG-based approaches addressing questions such as “what information to retrieve”, “when to retrieve” and “how to use the retrieved information” to ground LLMs in the medical domain.”
Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications
Bora, A., & Cuayáhuitl, H. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6040116 · Cited by 61
Zero-Shot Clinical Trial Patient Matching with LLMs
Wornow, M., Lozano, A., Dash, D., Jindal, J., Mahaffey, K. W., & Shah, N. H. (2025) · NEJM AI · doi:10.1056/aics2400360 · Cited by 59
Zero-Shot Clinical Trial Patient Matching with LLMs
Wornow, M., Lozano, A., Dash, D., Jindal, J., Mahaffey, K. W., & Shah, N. H. (2024) · arXiv · doi:10.48550/arxiv.2402.05125 · Cited by 59 · cited 2×
“Similarly, a recent study used two retrieval pipelines, first selecting the top-k most pertinent segments from the patients’ notes, and then using top-k segments as a prompt input to assess the LLM [108].”
Woo, J. J., Yang, A. J., Olsen, R. J., Hasan, S. S., Nawabi, D. H., Nwachukwu, B. U., Williams, R. J., & Ramkumar, P. N. (2025) · Arthroscopy: The Journal of Arthroscopic & Related Surgery · doi:10.1016/j.arthro.2024.10.042 · Cited by 57 · cited 1×
“Finally, the authors compared the output of the custom models to general pre-trained LLMs such as GPT-4 and LLaMA-2.”
Duan, Y., Zhou, Q., Yu, L., Qin, C., Wang, Z., Kan, H., & Hu, J. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1512329 · Cited by 56
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning
Luo, L., Li, Y.-F., Haffari, G., & Pan, S. (2023) · arXiv · doi:10.48550/arxiv.2310.01061 · Cited by 56 · cited 1×
“Compared to knowledge stored in unstructured documents such as in the portable document format (PDF), KGs are more ideal for RAG because of the easiness in accessing relations among knowledge items [132].”
Knowledge grounded medical dialogue generation using augmented graphs
Varshney, D., Zafar, A., Behera, N. K., & Ekbal, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-29213-8 · Cited by 55 · cited 1×
“Two conversational models, MedKgConv[85] and MED[86] for medical dialogue generation were proposed, utilising multi-head attention and knowledge-aware neural conversation respectively.”
Diving Deep into Modes of Fact Hallucinations in Dialogue Systems
Das, S., Saha, S., & Srihari, R. K. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.48 · Cited by 55
Lozano, A., Fleming, S. L., Chiang, C.-C., & Shah, N. (2023) · Biocomputing 2024 · doi:10.1142/9789811286421_0002 · Cited by 55
Garcia, B. T., Westerfield, L. E., Yelemali, P., Gogate, N., Rivera-Munoz, E. A., Du, H., Dawood, M., Jolly, A., Lupski, J. R., & Posey, J. E. (2025) · Genome Medicine · doi:10.1186/s13073-025-01521-w · Cited by 54
Do Language Models Plagiarize?
Lee, J., Le, T., Chen, J., & Lee, D. (2023) · Proceedings of the ACM Web Conference 2023 · doi:10.1145/3543507.3583199 · Cited by 51 · cited 1×
“Data leakage is another known problem for LLMs that has been extensively studied [101–103].”
Do Language Models Plagiarize?
Lee, J., Le, T., Chen, J., & Lee, D. (2022) · arXiv · doi:10.48550/arxiv.2203.07618 · Cited by 51 · cited 1×
“Data leakage is another known problem for LLMs that has been extensively studied [101–103].”
Russe, M. F., Fink, A., Ngo, H., Tran, H., Bamberg, F., Reisert, M., & Rau, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-41512-8 · Cited by 49 · cited 1×
“Similarly, Russe et al. [71] used LlamaIndex as an interface between the external knowledge and a context-aware LLM (FraCChat).”
Improving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model
Soong, D., Sridhar, S., Si, H., Wagner, J.-S., Sá, A. C. C., Yu, C. Y., Karagoz, K., Guan, M., Kumar, S., Hamadeh, H., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000568 · Cited by 49
Retrieval augmentation of large language models for lay language generation
Guo, Y., Qiu, W., Leroy, G., Wang, S., & Cohen, T. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104580 · Cited by 48 · cited 3×
“However, recently Chinese datasets such MedDG [29] and MedDialog-CN [28] have been proposed”
Chen, X., Wang, L., You, M., Liu, W., Fu, Y., Xu, J., Zhang, S., Chen, G., Li, K., & Li, J. (2024) · Journal of Medical Internet Research · doi:10.2196/58158 · Cited by 48 · cited 1×
“Though RAG substantially reduces the LLM’s ability to use memorised data from model training [72], it does not avert bias completely, which is baked into the LLMs underlying training data.”
Shi, W., Zhuang, Y., Zhu, Y., Iwinski, H., Wattenbarger, M., & Wang, M. D. (2023) · Proceedings of the 14th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics · doi:10.1145/3584371.3612956 · Cited by 47 · cited 1×
“Though RAG substantially reduces the LLM’s ability to use memorised data from model training [72], it does not avert bias completely, which is baked into the LLMs underlying training data.”
A RAG Chatbot for Precision Medicine of Multiple MyelomaPreprint
Quidwai, M., & Laganà, A. (2024) · medRxiv · doi:10.1101/2024.03.14.24304293 · Cited by 47 · cited 1×
“For example, Quidwai and Lagana [93] developed a system with a sophisticated retrieval component which allowed for the use of a predefined threshold to determine when insufficient relevant information were retrieved, after which the system can respond “Sorry, I could not find relevant information…”
Long, C., Subburam, D., Lowe, K., dos Santos, A., Zhang, J., Hwang, S., Saduka, N., Horev, Y., Su, T., Côté, D. W. J., et al. (2024) · Otolaryngology–Head and Neck Surgery · doi:10.1002/ohn.864 · Cited by 46 · cited 2×
“Similarly, a technique that flags safety concerns was developed, demonstrating zero instances of alarming red flags during testing [49].”
Knowledge graph assisted end-to-end medical dialog generation
Varshney, D., Zafar, A., Behera, N. K., & Ekbal, A. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102535 · Cited by 45 · cited 1×
“Two conversational models, MedKgConv[85] and MED[86] for medical dialogue generation were proposed, utilising multi-head attention and knowledge-aware neural conversation respectively.”
Das, S., Ge, Y., Guo, Y., Rajwal, S., Hairston, J., Powell, J. M., Walker, A. L., Peddireddy, S. R., Lakamana, S., Bozkurt, S., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/66220 · Cited by 45 · cited 1×
“A review of the latter approaches is provided in the next section.”
GastroBot: a Chinese gastrointestinal disease chatbot based on the retrieval-augmented generation
Zhou, Q., Liu, C., Duan, Y., Sun, K., Li, Y., Kan, H., Gu, Z., Shu, J., & Hu, J. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1392555 · Cited by 44 · cited 2×
“Modular RAG incorporates several techniques and modules from advanced RAG, allowing more customisation and optimization of the RAG system, as well as integration of methods to improve different functions [16].”
Steybe, D., Poxleitner, P. J., Aljohani, S., Herlofson, B. B., Nicolatou‐Galitis, O., Patel, V., Fedele, S., Kwon, T., Fusco, V., Pichardo, S. E. C., et al. (2025) · Journal of Cranio-Maxillofacial Surgery · doi:10.1016/j.jcms.2024.12.009 · Cited by 44
Ong, J. C. L., Jin, L., Elangovan, K., Lim, G. Y. S., Lim, D. Y. Z., Sng, G. G., Ke, Y., Tung, J. Y. M., Zhong, R. J., Koh, C. M. Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.01741 · Cited by 43
(2023) · doi:10.1101/2023.11.10.23298364 · Cited by 42 · cited 2×
“Ge et al. [47] used text embeddings to transform guidelines and guidance documents for liver diseases and conditions.”
Parmanto, B., Aryoyudanta, B., Soekinto, T. W., Setiawan, I. M. A., Wang, Y., Hu, H., Saptono, A., & Choi, Y. K. (2024) · JMIR Formative Research · doi:10.2196/54633 · Cited by 42
Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering
Wang, Y., Ma, X., & Chen, W. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.95 · Cited by 42
Bridging the Gap Between Consumers’ Medication Questions and Trusted Answers
Ben, A. A., Yassine, M., Mark, S., Goodwin Travis, R., Shooshan Sonya, E., & Dina, D.-F. (2019) · Studies in Health Technology and Informatics · doi:10.3233/shti190176 · Cited by 41
Ghadban, Y. A., Lu, H.-Y., Adavi, U., Sharma, A., Gara, S., Das, N., Kumar, B., John, R., Devarsetty, P., & Hirst, J. (2023) · medRxiv · doi:10.1101/2023.12.15.23300009 · Cited by 41 · cited 1×
“Various studies have indicated that RAG reduces hallucinations [19, 54,78,81,82,146].”
Ziletti, A., & D'Ambrosi, L. (2024) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2024.clinicalnlp-1.4 · Cited by 41
So, J.-H., Chang, J., Kim, E., Na, J., Choi, J., Sohn, J.-Y., Kim, B.-H., & Chu, S. H. (2024) · JMIR Formative Research · doi:10.2196/58418 · Cited by 36
Representation in AI Evaluations
Bergman, A. S., Hendricks, L. A., Rauh, M., Wú, B., Agnew, W. S., Kunesch, M., Duan, I., Gabriel, I., & Isaac, W. S. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594019 · Cited by 35 · cited 1×
“Representation is crucial for fairness and equity in medical AI systems [108,109].”
Retrieval-Augmented Generation with Graphs (GraphRAG)
Han, H., Wang, Y., Shomer, H., Guo, K., Ding, J., Lei, Y., Halappanavar, M., Rossi, R. A., Mukherjee, S., Tang, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.00309 · Cited by 34 · cited 1×
“Unlike the knowledge found in document repositories, KGs provide structured and inferable information, making them more suitable for augmenting LLMs-RAG [125,126].”
From RAGs to riches: utilizing large language models to write documents for clinical trials
Markey, N., El-Mansouri, I., Rensonnet, G., van Langen, C., & Meier, C. (2025) · Clinical Trials · doi:10.1177/17407745251320806 · Cited by 33 · cited 1×
“Data leakage is another known problem for LLMs that has been extensively studied [101–103].”
Kang, B., Kim, J., Yun, T.-R., & Kim, C.-E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.11246 · Cited by 32 · cited 2×
“In another study, a new approach to perform RAG is presented, eliminating the reliance on vector embedding by employing direct and flexible retrieval using natural language prompts [72].”
Lee, J., Cha, H., Hwangbo, Y., & Cheon, W. (2024) · Journal of Personalized Medicine · doi:10.3390/jpm14121131 · Cited by 31 · cited 1×
“In a nutshell, these concerns can be addressed by implementing robust data privacy measures, promoting transparency and accountability, mitigating bias, emphasizing human oversight, and promoting human-autonomy.”
Development and Testing of Retrieval Augmented Generation in Large Language ModelsPreprint
Ke, Y. H., Jin, L., Elangovan, K., Abdullah, H. R., Liu, N., Sia, A. T. H., Soh, C. R., Tung, J. Y. M., Ong, J. C. L., & Ting, D. S. W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4719185 · Cited by 30
Neupane, S., Mitra, S., Mittal, S., Gaur, M., Golilarz, N. A., Rahimi, S., & Amirlatifi, A. (2024) · ACM Transactions on Computing for Healthcare · doi:10.1145/3709365 · Cited by 30
Mashatian, S., Armstrong, D. G., Ritter, A., Robbins, J., Aziz, S., Alenabi, I., Huo, M., Anand, T., & Tavakolian, K. (2025) · Journal of Diabetes Science and Technology · doi:10.1177/19322968241253568 · Cited by 28
Vaid, A., Lampert, J. M., Lee, J., Sawant, A. S., Apakama, D. U., Sakhuja, A., Soroush, A., Bick, S., Abbott, E., Gomez, H., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.02851 · Cited by 26
Griewing, S., Lechner, F., Gremke, N., Lukac, S., Janni, W., Wallwiener, M., Wagner, U., Hirsch, M., & Kuhn, S. (2024) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-024-05964-3 · Cited by 26
RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01352 · Cited by 26 · cited 1×
“For example, modular RAG can include a search module for similarity retrieval, which can improve the quality and diversity of the retrieved content, and apply a fine-tuning approach in the retriever, which can adapt the retriever to the specific domain or task [130].”
Painter, J. L., Chalamalasetti, V. R., Kassekert, R., & Bate, A. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooaf003 · Cited by 24
Using Large Language Models to Retrieve Critical Data from Clinical Processes and Business Rules
Yu, Y., Gomez-Cabello, C. A., Makarova, S., Parte, Y., Borna, S., Haider, S. A., Genovese, A., Prabha, S., & Forte, A. J. (2024) · Bioengineering · doi:10.3390/bioengineering12010017 · Cited by 23
Predictive models to assess Oxygen Deficiency Hazard (ODH): A systematic review
Stefana, E., Marciano, F., Cocca, P., & Alberti, M. (2015) · Safety Science · doi:10.1016/j.ssci.2015.01.008 · Cited by 22 · cited 1×
“We used a three-step process for the selection of articles to be included in this study: relevance of the title, relevance of the abstract and finally relevance of the full-text [26].”
A Retrieval-augmented Generation application for Question-Answering in Nutrigenetics Domain
Benfenati, D., Filippis, G. M. D., Rinaldi, A. M., Russo, C., & Tommasino, C. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.467 · Cited by 21
Retrieval-Based Diagnostic Decision Support: Mixed Methods Study
Abdullahi, T., Mercurio, L., Singh, R., & Eickhoff, C. (2024) · JMIR Medical Informatics · doi:10.2196/50209 · Cited by 20
Citation-Enhanced Generation for LLM-based Chatbots
Li, W., Li, J., Ma, W., & Liu, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.79 · Cited by 20 · cited 1×
“Future studies should focus on techniques to robustly handle noise, information integration and improving the validity of sources, e.g., using post-hoc citation-enhanced generation [117].”
Bailicai: A Domain-Optimized Retrieval-Augmented Generation Framework for Medical ApplicationsPreprint
Long, C., Liu, Y., Ouyang, C.-P., & Yu, Y. (2024) · arXiv.org · doi:10.48550/arxiv.2407.21055 · Cited by 20 · cited 1×
“Various studies have indicated that RAG reduces hallucinations [19, 54,78,81,82,146].”
Zheng, Y., Yan, Y., Chen, S., Cai, Y., Ren, K., Liu, Y., Zhuang, J., & Zhao, M. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1501408 · Cited by 18
REMED: Retrieval-Augmented Medical Document Query Responding with Embedding Fine-Tuning
Pang, T., Tan, K., Yao, Y., Liu, X., Meng, F., Fan, C., & Zhang, X. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651011 · Cited by 17 · cited 2×
“In other studies, embedding fine-tuned was employed [91,117].”
Klang, E., Tessler, I., Apakama, D. U., Abbott, E. E., Glicksberg, B. S., Arnold, M., Moses, A., Sakhuja, A., Soroush, A., Charney, A. W., et al. (2024) · medRxiv · doi:10.1101/2024.10.15.24315526 · Cited by 15
PodGPT: an audio-augmented large language model for research and education
Jia, S., Bit, S., Searls, E., Lauber, M. V., Fan, P., Wang, W. M., Claus, L. A., Jasodanand, V. H., Veerapaneni, D., Au, R., et al. (2025) · npj Biomedical Innovations. · doi:10.1038/s44385-025-00022-0 · Cited by 15
A Prospective Comparison of Large Language Models for Early Prediction of Sepsis
Shashikumar, S. P., & Nemati, S. (2024) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789819807024_0009 · Cited by 13
Fukushima, T., Manabe, M., Yada, S., Wakamiya, S., Yoshida, A., Urakawa, Y., Maeda, A., Kan, S., Takahashi, M., & Aramaki, E. (2024) · JMIR Medical Informatics · doi:10.2196/65047 · Cited by 13
MedDialog: Two Large-scale Medical Dialogue Datasets
He, X., Chen, S., Ju, Z., Dong, X., Fang, H., Wang, S., Yang, Y., Zeng, J., Zhang, R., Zhang, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.03329 · Cited by 12 · cited 2×
“Among the available datasets, MedDialog [28] contains conversations and utterances between patients and doctors in both English and Chinese, sourced from two websites, namely healthcaremagic.com and iclinic.com.”
Long, C., Subburam, D., Lowe, K., et al. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4571725 · Cited by 10
FAVOR-GPT: a generative natural language interface to whole genome variant functional annotations
Li, T. C., Zhou, H., Verma, V., Tang, X., Shao, Y., Buren, E. V., Weng, Z., Gerstein, M., Neale, B. M., Sunyaev, S. R., et al. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae143 · Cited by 10
VPL: Visual Proxy Learning Framework for Zero-Shot Medical Image Diagnosis
Liu, J., Hu, T., Xiong, H., Du, J., Feng, Y., Wu, J., Zhou, J. T., & Liu, Z. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.583 · Cited by 10
Multimodal LLMs for health grounded in individual-specific dataPreprint
Belyaeva, A., Cosentino, J., Hormozdiari, F., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.09018 · Cited by 8 · cited 1×
“Beyond images, multimodal LLMs grounded in an individual’s specific data to estimate disease risk have been proposed [20].”
Enhancing Dietary Supplement Question Answer via Retrieval-Augmented Generation (RAG) with LLMPreprint
Hou, Y., & Zhang, R. (2024) · medRxiv · doi:10.1101/2024.09.11.24313513 · Cited by 8
Garcia, B. T., Westerfield, L. E., Yelemali, P., et al. (2024) · medRxiv · doi:10.1101/2024.12.01.24318253 · Cited by 8
Rau, A., Rau, S., Fink, A., et al. (2023) · medRxiv · doi:10.1101/2023.04.10.23288354 · Cited by 6 · cited 1×
“A recent review outlined the progression of RAG technologies and their impact on tasks requiring extensive knowledge [16].”
PodGPT: An audio-augmented large language model for research and educationPreprint
Jia, S., Bit, S., Searls, E., et al. (2024) · medRxiv · doi:10.1101/2024.07.11.24310304 · Cited by 5
Ke, Y., Jin, L., Elangovan, K., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-5062476/v1 · Cited by 4
Long, C., Subburam, D., Lowe, K., et al. (2023) · medRxiv · doi:10.1101/2023.08.18.23294283 · Cited by 2
Zhao, X., Liu, S., Yang, S.-Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.04413 · Cited by 2 · cited 2×
“Another key observation from our review is the language bias in the datasets used as external knowledge sources.”
Self-Alignment with Instruction Backtranslation
Li, X., Yu, P., Zhou, C., Schick, T., Levy, O., Zettlemoyer, L., Weston, J., & Lewis, M. (2023) · arXiv · doi:10.48550/arxiv.2308.06259 · cited 1×
“While iterative data refinement has been attempted for LLM training [154] and RAG in general domains [155,156], these methods have not yet been explored in the medical field.”
GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning
Solatorio, A. V. (2024) · arXiv · doi:10.48550/arxiv.2402.16829
Wu, J., Zhu, J., Qi, Y., Chen, J., Xu, M., Menolascina, F., & Grau, V. (2024) · arXiv · doi:10.48550/arxiv.2408.04187
RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering
Arasteh, S. T., Lotfinia, M., Bressem, K., Siepmann, R., Adams, L., Ferber, D., Kuhl, C., Kather, J. N., Nebelung, S., & Truhn, D. (2024) · arXiv · doi:10.48550/arxiv.2407.15621 · cited 2×
“For instance, Aratesh et al. [54] found that GPT-4 exhibited the fewest hallucinations in their study.”
Research works printed but not linked: 4
- Abacha AB, Agichtein E, Pinter Y, Demner-Fushman D. Overview of the medical question answering task at TREC 2017 LiveQA. In: Text Retrieval Conference; 2017. Available from: https://api.semanticscholar.org/CorpusID:3902472
- Wang J, Yang Z, Yao Z, Yu H. JMLR: joint medical LLM and retrieval training for enhancing reasoning and professional question answering capability. 2024.
- H Yu, 2023, Zero-shot ECG diagnosis with large language models and retrieval-augmented generation., 650
- DD Team, 2024, DeepEval: the open-source LLM evaluation framework
Cited by
Retrieval-Augmented Generation (RAG) Chatbots for Education: A Survey of Applications
Swacha, J., & Gracel, M. (2025) · Applied Sciences · doi:10.3390/app15084234 · Cited by 244
Hindi, M., Mohammed, L., Maaz, O., & Alwarafy, A. (2025) · IEEE Access · doi:10.1109/access.2025.3550145 · Cited by 140
AI Agents in Clinical Medicine: A Systematic ReviewPreprint
Gorenshtein, A., Omar, M., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025) · medRxiv · doi:10.1101/2025.08.22.25334232 · Cited by 94
“14,15 20, 21, 22 Unlike standard”
The rise of small language models in healthcare: A comprehensive survey
Garg, M., Raza, S., Rayana, S., Liu, X., & Sohn, S. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.100999 · Cited by 81
A Current Review of Generative AI in Medicine: Core Concepts, Applications, and Current Limitations
Rouzrokh, P., Khosravi, B., Faghani, S., Moassefi, M., Shariatnia, M. M., Rouzrokh, P., & Erickson, B. J. (2025) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-025-09961-y · Cited by 66
“RAG models provide significant advancements in over - coming the limitations of traditional large language models (LLMs), where accuracy and timeliness are critical [ 83 ].”
Metadata
- Authors
- Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J.
- Affiliations
- Boehringer Ingelheim (Germany) (DE)
- Boehringer Ingelheim (China) (CN)
- Corresponding author
- Amugongo, Lameck Mbangula
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2025-06-11
- Volume
- 4
- Issue
- 6
- Pages
- e0000877
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC12157099
- OpenAlex
- W4411203672
- Semantic Scholar
- 5879575701b9b65b5cc56c00d9eebbfa219e0428
- Cited by
- 267
- References
- 126
- Citations by year
- 2026: 218 · 2025: 48
- FWCI
- 15.74
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- retrieval-augmented generation, large language models, health care, medical question answering, hallucinations, external knowledge sources, medical datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.