CoChat

Retrieval augmented generation for large language models in healthcare: A systematic review

Open access
Amugongo, Lameck Mbangula, Mascheroni, Pietro, Brooks, Steven, Doering, Stefan, Seidel, Jan · 2025 · PLOS Digital Health

The DOI matches its Crossref record; 161 of the 161 references it cites were found in a registry or an index.

Abstract

Large Language Models (LLMs) have demonstrated promising capabilities to solve complex tasks in critical sectors such as healthcare. However, LLMs are limited by their training data which is often outdated, the tendency to generate inaccurate (“hallucinated”) content and a lack of transparency in the content they generate. To address these limitations, retrieval augmented generation (RAG) grounds the responses of LLMs by exposing them to external knowledge sources. However, in the healthcare domain there is currently a lack of systematic understanding of which datasets, RAG methodologies and evaluation frameworks are available. This review aims to bridge this gap by assessing RAG-based approaches employed by LLMs in healthcare, focusing on the different steps of retrieval, augmentation and generation. Additionally, we identify the limitations, strengths and gaps in the existing literature. Our synthesis shows that 78.9% of studies used English datasets and 21.1% of the datasets are in Chinese. We find that a range of techniques are employed RAG-based LLMs in healthcare, including Naive RAG, Advanced RAG, and Modular RAG. Surprisingly, proprietary models such as GPT-3.5/4 are the most used for RAG applications in healthcare. We find that there is a lack of standardised evaluation frameworks for RAG-based applications. In addition, the majority of the studies do not assess or address ethical considerations related to RAG in healthcare. It is important to account for ethical challenges that are inherent when AI systems are implemented in the clinical setting. Lastly, we highlight the need for further research and development to ensure responsible and effective adoption of RAG in the medical domain.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pdig.0000877

Cites

161 of 161 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 PRISMA [25] workflow applied to the search, identification and selection of the studies that were included in the systematic review.
Fig 1 PRISMA [25] workflow applied to the search, identification and selection of the studies that were included in the systematic review.CC BY · PubMed Central
Fig 2 A schematic illustration of the basic RAG workflow applied to answering a user question in the medical domain.
Fig 2 A schematic illustration of the basic RAG workflow applied to answering a user question in the medical domain.CC BY · PubMed Central
Fig 3 Metrics used to evaluate RAG-based systems in the medical domain. Additionally, we assess whether ethical principles: privacy, safety, robustness (robust), bias, trust (explainability/interpretability) have been considered.
Fig 3 Metrics used to evaluate RAG-based systems in the medical domain. Additionally, we assess whether ethical principles: privacy, safety, robustness (robust), bias, trust (explainability/interpretability) have been considered.CC BY · PubMed Central
Fig 4 a) Dataset Languages. b) LLMs explored for retrieval-augmented generation in healthcare. Please note that some studies used more than one model, hence the total count of models is higher than the number of studies included in this review.
Fig 4 a) Dataset Languages. b) LLMs explored for retrieval-augmented generation in healthcare. Please note that some studies used more than one model, hence the total count of models is higher than the number of studies included in this review.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 4

  1. Abacha AB, Agichtein E, Pinter Y, Demner-Fushman D. Overview of the medical question answering task at TREC 2017 LiveQA. In: Text Retrieval Conference; 2017. Available from: https://api.semanticscholar.org/CorpusID:3902472
  2. Wang J, Yang Z, Yao Z, Yu H. JMLR: joint medical LLM and retrieval training for enhancing reasoning and professional question answering capability. 2024.
  3. H Yu, 2023, Zero-shot ECG diagnosis with large language models and retrieval-augmented generation., 650
  4. DD Team, 2024, DeepEval: the open-source LLM evaluation framework

Cited by

See all 367

Metadata

Authors
Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J.
Affiliations
  • Boehringer Ingelheim (Germany) (DE)
  • Boehringer Ingelheim (China) (CN)
Corresponding author
Amugongo, Lameck Mbangula
Venue
PLOS Digital Health
Published
2025-06-11
Volume
4
Issue
6
Pages
e0000877
ISSN
2767-3170 (ISSN-L)
OpenAlex
W4411203672
Cited by
267
References
126
Citations by year
2026: 218 · 2025: 48
FWCI
15.74
Type
journal article
Language
en
Open access
Locations
  • PLOS Digital Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
retrieval-augmented generation, large language models, health care, medical question answering, hallucinations, external knowledge sources, medical datasets

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.