RAG in Health Care: A Novel Framework for Improving Communication and Decision-Making by Addressing LLM Limitations
Abstract
AbstractWithin the field of artificial intelligence (AI), large language models (LLMs) have the potential to transform the delivery of medical information. LLMs, as a subset of generative AI, have demonstrated value in content creation, idea generation, and interactive communication. However, their inherent limitations, such as the need for up-to-date information, hallucinations of incorrect facts, and a reliance on public-domain data, restrict the full potential of generative AI within the health care setting. To address these limitations, retrieval-augmented generation (RAG) offers a novel framework by connecting LLMs with external knowledge, enabling them to access information beyond their training data. Within the health care domain, additional datasets could include peer-reviewed studies, gated medical compendiums, and the internal policies of health care organizations such as hospitals or pharmaceutical companies. By leveraging RAG, existing generative AI tools gain the capability to consider both public and private information, expanding their application and enhancing accuracy and relevance within the health care setting. The utility of RAG in the health care setting has yet to be fully explored, but it has the potential to revolutionize the industry. This article seeks to outline present and future use cases of RAG for health care information exchange within both clinical and industrial settings.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Retrieval noise is a persistent challenge for RAG in healthcare.
“Persistent challenges are identified, including retrieval noise (irrelevant or low-quality retrieved information), domain shift (performance degradation when models are applied to data distributions different from their training set), generation latency, and limited explainability.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [9]
Domain shift is a persistent challenge for RAG in healthcare.
“Persistent challenges are identified, including retrieval noise (irrelevant or low-quality retrieved information), domain shift (performance degradation when models are applied to data distributions different from their training set), generation latency, and limited explainability.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [9]
Limited explainability is a persistent challenge for RAG in healthcare.
“Persistent challenges are identified, including retrieval noise (irrelevant or low-quality retrieved information), domain shift (performance degradation when models are applied to data distributions different from their training set), generation latency, and limited explainability.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [9]
Generation latency is a persistent challenge for RAG in healthcare.
“Persistent challenges are identified, including retrieval noise (irrelevant or low-quality retrieved information), domain shift (performance degradation when models are applied to data distributions different from their training set), generation latency, and limited explainability.”
From Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review (2025) · cites this paper as [9]
In rare rheumatic conditions, retrieval may break down because pertinent evidence is limited, hard to locate, or mismatched to how the clinical question is formulated.
“In rare rheumatic conditions, though, retrieval may break down because pertinent evidence is limited, hard to locate, or mismatched to how the clinical question is formulated [20–22].”
From Diagnostic performance of Prof. Valmed, ChatGPT-5 Thinking, and OpenEvidence in rheumatology: A comparative evaluation (2026) · cites this paper as [22]
Cites
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · see citing works
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463 · see citing works
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · see citing works
Almanac — Retrieval-Augmented Language Models for Clinical Medicine
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., et al. (2024) · NEJM AI · doi:10.1056/aioa2300068 · Cited by 424 · see citing works
Zaretsky, J., Kim, J. M., Baskharoun, S., Zhao, Y., Austrian, J., Aphinyanaphongs, Y., Gupta, R., Blecker, S. B., & Feldman, J. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.0357 · Cited by 303 · see citing works
Large language models to identify social determinants of health in electronic health records
Guevara, M., Chen, S., Thomas, S., Chaunzwa, T. L., Franco, I., Kann, B. H., Moningi, S., Qian, J. M., Goldstein, M., Harper, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00970-0 · Cited by 278 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review
Neha, F., Bhati, D., & Shukla, D. K. (2025) · AI · doi:10.3390/ai6090226 · Cited by 117 · see citing works
“…of condition-specific guidelines), domain shift (e.g., mismatched terminology between EHRs and biomedical literature), inference latency (e.g., delays in generating responses during emergencies), and limited interpretability (e.g., unclear how retrieved text shapes treatment suggestions) [9].”
Feng, X., Tian, L., Ho, G. W. K., Yorke, J., & Hui, V. (2025) · Journal of Medical Internet Research · doi:10.2196/79850 · Cited by 51 · see citing works
Workum, J. D., van de Sande, D., Gommers, D., & van Genderen, M. E. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1504805 · Cited by 25 · see citing works
Kremer, P., Langballe, E., Haase, I., Bamberger, J., Kuhn, S., Krusche, M., & Knitza, J. (2026) · Rheumatology International · doi:10.1007/s00296-025-06068-y · Cited by 17 · see citing works
Vladika, J., Domres, A., Nguyen, M., Moser, R., Nano, J., Busch, F., Adams, L. C., Bressem, K. K., Bernhardt, D., Combs, S. E., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.24830 · Cited by 1 · see citing works
The economics of accuracy for medical reasoning with large language models
Bhattacharyya, K., & Kamabattula, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001182
- Authors
- Ng, K. K. Y., Matsuba, I., & Zhang, P. C.
- Affiliations
- Novo Nordisk Canada (Canada) (CA)
- OMERS, Toronto, ON, Canada
- University of Toronto (CA)
- Venue
- NEJM AI
- Publisher
- Massachusetts Medical Society
- Published
- 2025-01
- Volume
- 2
- Issue
- 1
- Pages
- AIra2400380-AIra2400380
- ISSN
- 2836-9386 (ISSN-L)
- OpenAlex
- W4405728682
- Cited by
- 107
- References
- 7
- Citations by year
- 2026: 52 · 2025: 55
- FWCI
- 2.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- NEJM AI (publishedVersion, public-domain)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- retrieval-augmented generation, large language models, health care, clinical decision-making, clinical communication, LLM limitations, artificial intelligence in healthcare
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
