Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework
Open accessThe DOI matches its Crossref record; 74 of the 91 references it cites were found in a registry or an index, and 10 are not checked yet; 35 of the 47 sources its claims cite were checked for support.
Abstract
Evaluation was conducted on PubMedQA, MedMCQA, and MedQA-US using hybrid retrieval combining BM25 and MedCPT with GPT-4o and BioGPT; the hybrid retriever achieved 92.14% recall, 74.36% precision, and 82.30% F1, and GPT-4o with hybrid retrieval reached 89.4% faithfulness on PubMedQA.
Verified 2026-11-19 · 10 references not checked yet
Overview
CoChat Summary
## TL;DR
This paper proposes a modular retrieval-augmented generation (RAG) framework for medical question answering that combines BM25 and MedCPT retrieval with GPT-4o and BioGPT generation. On three healthcare QA datasets, the hybrid retriever outperformed single retrievers, and GPT-4o generally produced more faithful and better-balanced answers than BioGPT.
## Key findings
- The proposed hybrid retriever achieved 92.14% context recall, 74.36% context precision, and 82.30% F1-score, outperforming BM25 (82.76% recall, 67.77% precision, 74.52% F1) and MedCPT (73.31% recall, 48.75% precision, 58.56% F1).
- On PubMedQA, GPT-4o with hybrid retrieval reached 89.4% faithfulness, 82.7% answer relevancy, 93.1% context precision, 80.6% context recall, and 88.0% BERTScore F1.
- On MedMCQA, GPT-4o achieved 78.1% answer relevancy and 77.9% context recall; on MedQA-US, it reached 34.7% faithfulness and 77.3% context recall.
- BioGPT had very high context precision on PubMedQA (98.4%) but much lower faithfulness (32.3%), suggesting strong lexical overlap without consistently reliable grounding.
## Methods
- The study used a modular RAG pipeline with four phases: data preparation and indexing, hybrid retrieval, response generation, and evaluation.
- It evaluated three healthcare QA datasets: PubMedQA, MedMCQA, and MedQA-US; the text also says retrieval evaluation used a 15,000-question corpus, and generation evaluation was done on 3,000 questions sampled evenly from the three datasets.
- Retrieval compared BM25, MedCPT, and a hybrid late-fusion approach that merged both retrievers; generation compared GPT-4o (few-shot prompting, temperature = 0, max_tokens = 512) and BioGPT (beam search).
- Evaluation used context precision, context recall, F1, Average Precision, plus RAGAS and BERTScore for generation quality.
## Limitations
- The paper explicitly states that it does not perform disease-specific or organ-level stratified analysis, so performance differences across subdomains such as cardiology, nephrology, and oncology were not separately examined.
- It also states that future work could extend to subdomain-aware evaluation; this makes the current evaluation broad rather than specialty-specific.
- Evident from the text: the framework was tested on only three datasets, and the reported generation evaluation was sampled from 3,000 questions, so the results may not cover all medical QA settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
19 Valid · 7 Not valid · 10 Not checked · 96 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“By incorporating external information retrieval, RAG enhances generative models with improved accuracy and credibility for knowledge-intensive tasks.”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 1. Introduction
What the sources say
- Supportedcites [8] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., and Wang, H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv. source
The Introduction (page not specified in our copy) says RAG enhances LLMs by retrieving information from external knowledge bases and reduces factually incorrect content. It identifies knowledge-intensive tasks as a setting where LLMs face limitations; the Overview likewise describes retrieval helping them produce well-informed answers. These passages support the claim’s accuracy and credibility paraphrase. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Most existing Modular RAG systems in healthcare, such as Self-BioRAG [16] and Clinfo.ai[24], rely solely on either sparse (BM25) or dense (MedCPT [25], embedding-based) retrieval methods.”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 2. Related Works
What the sources say
- Supportedcites [16] Jeong, 2024, Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models, Bioinformatics source
The citation identifies Self-BioRAG as an example. Section 3.2, “Biomedical retriever,” says Self-BioRAG uses MedCPT to retrieve evidence, followed by reranking. This supports the retrieval-method attribution for Self-BioRAG; this citation is not offered as evidence for Clinfo.ai or for the prevalence claim about most systems. (LLM, conf=0.90)
- Not checked yetcites [24] Lozano, A., Fleming, S.L., Chiang, C.C., and Shah, N. (2024, January 3–7). Clinfo.ai: An open-source retrieval-augmented large language model system for answering medical questions using scientific literature. Proceedings of the Pacific Symposium on Biocomputing 2024, Honolulu, HI, USA. source
Checked before — its earlier verdict is held back until a new check runs.
- Supportedcites [25] Jin, 2023, MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval, Bioinformatics source
The citation is attached specifically to MedCPT, identifying it as a dense, embedding-based retrieval method. Section 2.2 describes its query and document encoders and nearest-neighbor search; the Introduction explains that dense retrievers encode queries and documents as vectors. The citation does not attach to the broader assertion about most Modular RAG systems. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, current RAG models face challenges related to frequent knowledge updates [2,11,21,22].”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 1. Introduction
What the sources say
- Could not verifycites [2] Yang, 2023, Large language models in health care: Development, applications, and challenges, Health Care Sci. sourceExperimental — accuracy not yet measured
The 'Questionable credibility and accuracy of information' section discusses inaccurate LLM outputs, but not retrieval-augmented generation or difficulties with frequent knowledge updates. Searches for RAG, retrieval-augmented generation, and updated or outdated knowledge found no passage addressing the attributed claim; no page number is available in our copy. (LLM, conf=0.75)
- Partly supportedcites [11] Li, 2024, A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges, Vicinagearth sourceExperimental — accuracy not yet measured
Section 3.3 describes RAG as retrieving information from external knowledge sources and says such approaches still fall short for more profound knowledge updates. It also discusses the difficulty of keeping LLM knowledge current, but does not specifically say that frequent updates pose a challenge for RAG models. (LLM, conf=0.82)
- Partly supportedcites [21] Huang, J., Wang, M., Cui, Y., Liu, J., Chen, L., Wang, T., and Wu, J. (2024). Layered Query Retrieval: An Adaptive Framework for Retrieval-Augmented Generation in Complex Question Answering for Large Language Models. Appl. Sci., 14. sourceExperimental — accuracy not yet measured
Our copy does state that current RAG approaches carry unresolved limitations, but the challenges it enumerates are query-complexity handling, efficiency vs. accuracy, computational cost of iterative retrieval, and inconsistent/contradictory information when merging retrieved documents (Related Work; Introduction) — never the freshness or updating of knowledge. On knowledge currency the paper points the other way: it attributes the 'up-to-date knowledge'/knowledge-cutoff problem to pre-trained LLMs relying on static data and presents RAG as mitigating it (Introduction). So the 'current RAG models face challenges' part is supported, while the specific qualifier 'related to frequent knowledge updates' is not attributable to this source. (LLM, conf=0.68)
- Partly supportedcites [22] Xu, K., Zhang, K., Li, J., Huang, W., and Wang, Y. (2024). CRP-RAG: A Retrieval-Augmented Generation Framework for Supporting Complex Logical Reasoning and Knowledge Planning. Electronics, 14. sourceExperimental — accuracy not yet measured
Our copy does state that current RAG research faces challenges — the Introduction ('However, current RAG research still faces two issues') and Section 1 ('the RAG framework still faces two major challenges') — but the challenges it identifies are irrelevant-knowledge interference with reasoning, linear reasoning structures that block complex thought transformations, and the decoupling of knowledge retrieval/planning from the reasoning process, not frequent knowledge updates. No passage in our copy (searched: knowledge updates, outdated/temporal knowledge, knowledge-base refresh, dynamic/static knowledge modeling) attributes a knowledge-update difficulty to current RAG models; the closest text, Section 6.3, concerns CRP-RAG's own lack of 'autonomous evolution and coordinated updates based on environmental changes', i.e., a limitation of the proposed framework rather than of current RAG generally. Thus the generic frame of the claim is supported while the specific qualifier ('related to frequent knowledge updates') is not. (LLM, conf=0.62)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This pattern, observed in approximately 25% of cases, reflects instruction-following limitations where RAG systems struggle to align response style with the specific demands of the question type [52,60].”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 4.1. Error Analysis
What the sources say
- Partly supportedcites [52] Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2024, January 17–22). RAGAS: Automated Evaluation of Retrieval Augmented Generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, St. Julian’s, Malta. sourceExperimental — accuracy not yet measured
The Answer relevance section (page unavailable) says the metric assesses whether an answer appropriately addresses and aligns with the initial question or instruction. It does not characterize mismatches as instruction-following limitations or discuss response style and the demands of different question types. The cited paper also does not establish the approximately 25% figure, though that may describe the citing authors’ own observations. (LLM, conf=0.84)
- Partly supportedcites [60] Schmidgall, 2024, Evaluation and mitigation of cognitive biases in medical language models, npj Digit. Med. sourceExperimental — accuracy not yet measured
Methods > Automated evaluation (page unavailable) reports that two medical language models frequently ignored an instruction to answer with only a single letter, supporting the general point about instruction-following and response format. It does not establish a roughly 25% rate or examine RAG systems matching response style to different question types. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“At the retrieval stage, three primary error types were observed: context recall error [52], faithfulness error [52], and answer relevance mismatch [52].”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 4.1. Error Analysis
What the sources say
- Partly supportedcites [52] Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2024, January 17–22). RAGAS: Automated Evaluation of Retrieval Augmented Generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, St. Julian’s, Malta. sourceExperimental — accuracy not yet measured
Section 3, “Evaluation Strategies” (page unavailable), identifies three quality aspects: faithfulness, answer relevance, and context relevance—not context recall. It defines faithfulness and answer relevance in terms of the generated answer, so the source does not support describing all three as error types observed at the retrieval stage. (LLM, conf=0.88)
plainn = 30- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
67 more claims · 7 with their cited sources checked
Create a free account to see all 132 claims and to verify the 5 references still open.
- It may state two separate claims as one. 38
- Probably not a substantial claim. 40
- Borderline: one of the tests for a substantial claim is unclear. 1
Citations
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Probable Inference, the Law of Succession, and Statistical Inference
Wilson, E. B. (1927) · Journal of the American Statistical Association · doi:10.1080/01621459.1927.10502953 · Cited by 4,476
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D. S., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1705.03551 · Cited by 1,650
National Standards for Diabetes Self-Management Education
Funnell, M. M., Brown, T. L., Childs, B. P., Haas, L. B., Hosey, G. M., Jensen, B., Maryniuk, M., Peyrot, M., Piette, J. D., Reader, D., et al. (2009) · Diabetes Care · doi:10.2337/dc09-s087 · Cited by 1,350
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., & Liu, T.-Y. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac409 · Cited by 1,150
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718
PubMedQA: A Dataset for Biomedical Research Question AnsweringPreprint
Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W., & Lu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06146 · Cited by 714
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685
RAGAs: Automated Evaluation of Retrieval Augmented Generation
Es, S., James, J., Espinosa Anke, L., & Schockaert, S. (2024) · Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2024.eacl-demo.16 · Cited by 651
A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024) · Vicinagearth · doi:10.1007/s44336-024-00009-2 · Cited by 404
Large language models in health care: Development, applications, and challenges
Yang, R., Tan, T. F., Lu, W., Thirunavukarasu, A. J., Ting, D. S. W., & Liu, N. (2023) · Health Care Science · doi:10.1002/hcs2.61 · Cited by 366
Gupta, S., Ranjan, R., & Singh, S. N. (2024) · arXiv · doi:10.48550/arxiv.2410.12837 · Cited by 342
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337
Benchmarking Retrieval-Augmented Generation for Medicine
Xiong, G., Jin, Q., Lu, Z., & Zhang, A. (2024) · arXiv · doi:10.48550/arxiv.2402.13178 · Cited by 312
Benchmarking Retrieval-Augmented Generation for Medicine
Xiong, G., Jin, Q., Lu, Z., & Zhang, A. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.372 · Cited by 310
Retrieval-augmented generation for generative artificial intelligence in health care
Yang, R., Ning, Y., Keppo, E., Liu, M., Hong, C., Bitterman, D. S., Ong, J. C. L., Ting, D. S. W., & Liu, N. (2025) · npj Health Systems · doi:10.1038/s44401-024-00004-1 · Cited by 269
Retrieval augmented generation for large language models in healthcare: A systematic review
Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000877 · Cited by 267
Cascella, M., Semeraro, F., Montomoli, J., Bellini, V., Piazza, O., & Bignami, E. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02045-3 · Cited by 219
Re2G: Retrieve, Rerank, GeneratePreprint
Glass, M. R., Rossiello, G., Chowdhury, M. F. M., Naik, A. R., Cai, P., & Gliozzo, A. (2022) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2207.06300 · Cited by 183
Jin, Q., Kim, W., Chen, Q., Comeau, D. C., Yeganova, L., Wilbur, W. J., & Lu, Z. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad651 · Cited by 175
Evaluation of Retrieval-Augmented Generation: A Survey
Yu, H., Gan, A., Zhang, K., Tong, S., Liu, Q., & Liu, Z. (2025) · Communications in Computer and Information Science · doi:10.1007/978-981-96-1024-2_8 · Cited by 173
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2022) · arXiv · doi:10.48550/arxiv.2203.14371 · Cited by 157
Alkhalaf, M., Yu, P., Yin, M., & Deng, C. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104662 · Cited by 156
Sawarkar, K., Mangal, A., & Solanki, S. R. (2024) · 2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr62202.2024.00031 · Cited by 145
Medical Hallucination in Foundation Models and Their Impact on Healthcare
Kim, Y., Jeong, H., Chen, S., Li, S. S., Park, C., Lu, M., Alhamoud, K., Mun, J., Grau, C., Jung, M., et al. (2025) · medRxiv · doi:10.1101/2025.02.28.25323115 · Cited by 136
Jeong, M., Sohn, J., Sung, M., & Kang, J. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae238 · Cited by 132
A survey on retrieval-augmentation generation (RAG) models for healthcare applications
El-Enen, M. A. A., Saad, S., & Nazmy, T. M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11666-9 · Cited by 118
Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review
Neha, F., Bhati, D., & Shukla, D. K. (2025) · AI · doi:10.3390/ai6090226 · Cited by 117
Health-LLM: Personalized Retrieval-Augmented Disease Prediction SystemPreprint
Yu, Q., Jin, M., Shu, D., Zhang, C., Fan, L., Hua, W., Zhu, S., Meng, Y., Wang, Z., Du, M., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.00746 · Cited by 97
Large language models in biomedicine and health: current research landscape and future directions
Lu, Z., Peng, Y., Cohen, T., Ghassemi, M., Weng, C., & Tian, S. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae202 · Cited by 92
Telco-RAG: Navigating the Challenges of Retrieval Augmented Language Models for Telecommunications
Bornea, A.-L., Ayed, F., Domenico, A. D., Piovesan, N., & Maatouk, A. (2024) · Globecom · doi:10.1109/globecom52923.2024.10901158 · Cited by 80
Re2G: Retrieve, Rerank, Generate
Glass, M., Rossiello, G., Chowdhury, M. F. M., Naik, A., Cai, P., & Gliozzo, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.194 · Cited by 79
RAGAs: Automated Evaluation of Retrieval Augmented Generation
Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2023) · arXiv · doi:10.48550/arxiv.2309.15217 · Cited by 79
Evaluation and mitigation of cognitive biases in medical language models
Schmidgall, S., Harris, C., Essien, I., Olshvang, D., Rahman, T., Kim, J. W., Ziaei, R., Eshraghian, J., Abadir, P., & Chellappa, R. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01283-6 · Cited by 68
Reasoning with large language models for medical question answering
Lucas, M. M., Yang, J., Pomeroy, J. K., & Yang, C. C. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae131 · Cited by 66
National Standards for Diabetes Self-Management Education
Funnell, M. M., Brown, T. L., Childs, B. P., Haas, L. B., Hosey, G. M., Jensen, B., Maryniuk, M. D., Peyrot, M. F., Piette, J. D., Reader, D., et al. (2007) · The Science of Diabetes Self-Management and Care · doi:10.1177/0145721707305880 · Cited by 63
MedExpQA: Multilingual benchmarking of Large Language Models for Medical Question Answering
Alonso, I., Oronoz, M., & Agerri, R. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102938 · Cited by 62
Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam GenerationPreprint
Guinet, G., Omidvar-Tehrani, B., Deoras, A., & Callot, L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.13622 · Cited by 59
Faithful AI in Medicine: A Systematic Review with Large Language Models and Beyond
Xie, Q., Schenck, E. J., Yang, H. S., Chen, Y., Peng, Y., & Wang, F. (2023) · doi:10.21203/rs.3.rs-3661764/v1 · Cited by 57
Lozano, A., Fleming, S. L., Chiang, C.-C., & Shah, N. (2023) · Biocomputing 2024 · doi:10.1142/9789811286421_0002 · Cited by 55
Xu, K., Zhang, K., Li, J., Huang, W. D., & Wang, Y. (2024) · Electronics · doi:10.3390/electronics14010047 · Cited by 51
Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model
Muludi, K., Fitria, K. M., Triloka, J., & Sutedi, S. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150379 · Cited by 50
GastroBot: a Chinese gastrointestinal disease chatbot based on the retrieval-augmented generation
Zhou, Q., Liu, C., Duan, Y., Sun, K., Li, Y., Kan, H., Gu, Z., Shu, J., & Hu, J. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1392555 · Cited by 44
Miao, Y., Zhao, Y., Luo, Y., Wang, H., & Wu, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/80557 · Cited by 38
Medical Hallucinations in Foundation Models and Their Impact on Healthcare
Kim, Y., Jeong, H., Chen, S., Li, S. S., Park, C., Lu, M., Alhamoud, K., Mun, J., Grau, C., Jung, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.05777 · Cited by 33
Mashatian, S., Armstrong, D. G., Ritter, A., Robbins, J., Aziz, S., Alenabi, I., Huo, M., Anand, T., & Tavakolian, K. (2025) · Journal of Diabetes Science and Technology · doi:10.1177/19322968241253568 · Cited by 28
Xu, K., Zhang, K., Li, J., Huang, W., & Wang, Y. (2024) · doi:10.20944/preprints202411.1648.v1 · Cited by 28
Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review
Neha, F., Bhati, D., & Shukla, D. K. (2025) · doi:10.20944/preprints202508.1022.v1 · Cited by 28
Bunnell, D. J., Bondy, M. J., Fromtling, L. M., Ludeman, E. M., & Gourab, K. (2025) · medRxiv · doi:10.1101/2025.04.01.25325033 · Cited by 22
Ammar, A., Koubaa, A., Nacar, O., & Boulila, W. (2026) · Studies in computational intelligence · doi:10.48550/arxiv.2505.08445 · Cited by 19
Huang, J., Wang, M., Cui, Y., Liu, J., Chen, L., Wang, T., Li, H., & Wu, J. (2024) · Applied Sciences · doi:10.3390/app142311014 · Cited by 17
Leveraging Large Language Models for Enhancing Literature-Based Discovery
Taleb, I., Navaz, A. N., & Serhani, M. A. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8110146 · Cited by 17
Retrieval Augmented Generation for Large Language Models in Healthcare: A Systematic ReviewPreprint
Amugongo, L. M., Mascheroni, P., Brooks, S. G., Doering, S., & Seidel, J. (2024) · Preprints.org · doi:10.20944/preprints202407.0876.v1 · Cited by 15
Omar, M., Sorin, V., Collins, J. D., Reich, D., Freeman, R., Gavin, N. P., Charney, A. W., Stump, L. S., Bragazzi, N. L., Nadkarni, G. N., et al. (2025) · medRxiv · doi:10.1101/2025.03.18.25324184 · Cited by 13
Detection, Diagnosis, and Explanation: A Benchmark for Chinese Medical Hallucination Evaluation
Dou, C., Zhang, Y., Chen, Y., Jin, Z., Jiao, W., Zhao, H., Zhao, Y., Tao, Z., & Huang, Y. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3jpikqxueoo2 · Cited by 13
Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks
Gao, Y., Xiong, Y., Wang, M., & Wang, H. (2024) · arXiv · doi:10.48550/arxiv.2407.21059 · Cited by 12
Retrieval-Augmented Generation for Generative Artificial Intelligence in MedicinePreprint
Yang, R., Ning, Y., Keppo, E., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.12449 · Cited by 6
Li, H., Lan, T., Fu, Z., Cai, D., Liu, L., Collier, N., Watanabe, T., & Su, Y. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3187 · Cited by 4
Shao, Y., Tang, X., Sohn, J., Chen, J., Liao, Y., Zhang, J., Xiang, J., Wu, F., Zhao, Y., Wu, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.07459 · Cited by 3
A Benchmark for Long-Form Medical Question Answering
Hosseini, P., Sin, J. M., Ren, B., Thomas, B. G., Nouri, E., Farahanchi, A., & Hassanpour, S. (2024) · arXiv · doi:10.48550/arxiv.2411.09834 · Cited by 1
Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for TelecommunicationsPreprint
Bornea, A.-L., Ayed, F., Domenico, A. D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.15939
Biomedical Literature Q&A System Using Retrieval-Augmented Generation (RAG)Preprint
Garg, M., Wang, L.-C., Ghanchi, B., Dumpala, S., Kakde, S., & Chen, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.05505
Miao, Y., Zhao, Y., Luo, Y., et al. (2025) · doi:10.2196/preprints.80557
Ammar, A., Koubaa, A., Nacar, O., & Boulila, W. (2026) · Studies in Computational Intelligence · doi:10.1007/978-3-032-24013-2_9
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., and Wang, H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv.
(n.d.)
UnresolvedIlin, I. (2025, May 02). Advanced RAG Techniques: An Illustrated Overview. Towards AI. Available online: https://pub.towardsai.net/advanced-rag-techniques-an-illustrated-overview-04d193d8fec6.
(n.d.)
UnresolvedRaja, M., and Yuvaraajan, E. (2024, January 24–26). A rag-based medical assistant especially for infectious diseases. Proceedings of the 2024 International Conference on Inventive Computation Technologies (ICICT), Lalitpur, Nepal.
(n.d.)
UnresolvedLewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., and Rocktäschel, T. (2020, January 6–12). Retrieval-augmented generation for knowledge-intensive NLP tasks. Proceedings of the 34th International Conference on Neural Information Processing S
(n.d.)
UnresolvedGarg, M., Wang, L.C., Ghanchi, B., Dumpala, S., Kakde, S., and Chen, Y.C. (2025). Biomedical Literature Q&A System Using Retrieval-Augmented Generation (RAG). arXiv.
(n.d.)
UnresolvedBajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., and Nguyen, T. (2016). Ms marco: A human generated machine reading comprehension dataset. arXiv.
(n.d.)
UnresolvedGuinet, G., Omidvar-Tehrani, B., Deoras, A., and Callot, L. (2024). Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation. arXiv.
(n.d.)
UnresolvedGao, Y., Xiong, Y., Wang, M., and Wang, H. (2024). Modular rag: Transforming rag systems into lego-like reconfigurable frameworks. arXiv.
(n.d.)
UnresolvedRetrieval-Augmented Generation Approach: Document Question Answering using Large Language Model
Muludi (2024) · Int. J. Adv. Comput. Sci. Appl.
UnresolvedGupta, S., Ranjan, R., and Singh, S.N. (2024). A comprehensive survey of retrieval-augmented generation (rag): Evolution, current landscape and future directions. arXiv.
(n.d.)
UnresolvedJin, M., Yu, Q., Shu, D., Zhang, C., Fan, L., Hua, W., and Zhang, Y. (2024). Health-LLM: Personalized Retrieval-Augmented Disease Prediction System. arXiv.
(n.d.)
UnresolvedPal, A., Umapathi, L.K., and Sankarasubbu, M. (2022, January 7–8). Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. Proceedings of the Conference on Health, Inference, and Learning, Virtual.
(n.d.)
UnresolvedHosseini, P., Sin, J.M., Ren, B., Thomas, B.G., Nouri, E., Farahanchi, A., and Hassanpour, S. (2024). A benchmark for long-form medical question answering. arXiv.
(n.d.)
UnresolvedTang, X., Shao, D., Sohn, J., Chen, J., Zhang, J., Xiang, J., Wu, F., Zhao, Y., Wu, C., and Shi, W. (2025). Medagentsbench: Benchmarking thinking models and agent frameworks for complex medical reasoning. arXiv.
(n.d.)
UnresolvedZhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2019). Bertscore: Evaluating text generation with bert. arXiv.
(n.d.)
UnresolvedGuo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017, January 6–11). On calibration of modern neural networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
(n.d.)
UnresolvedHoltzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y. (2019). The curious case of neural text degeneration. arXiv.
(n.d.)
Unresolved
Cited by
Monika, W., Dewi, D. A., Nasution, A. H., Onan, A., & Murakami, Y. (2026) · Information · doi:10.3390/info17070660 · Cited by 1
“Recent studies include hybrid medical RAG, knowledge-graph medical QA, clinical decision support, electronic health records, and evidence-based hallucination reduction [28,32,52,61,63,90,107,109].”
Development and Multireader Evaluation of Radiological RAG-System
Erizhokov, R. A., Gordeev, A. E., Sakharova, P. A., Yafarova, A. A., Varyukhina, M. D., Blokhin, I. A., Omelyanskaya, O. V., Vladzymyrskyy, A. V., & Vasilev, Y. A. (2026) · Data · doi:10.3390/data11060143 · Cited by 1
“This number was selected empirically as an optimal trade-off between coverage completeness and computational efficiency: a smaller number reduced contextual completeness, whereas a larger number increased noise and response time [27].”
Jamil, M., Kavak, A., Omurca, S. İ., & Fotouhi, H. (2026) · Diagnostics · doi:10.3390/diagnostics16193142
Metadata
- Authors
- Aljohani, B., & Alsanoosy, T.
- Affiliations
- Taibah University (SA)
- Corresponding author
- Alsanoosy, Tawfeeq
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2026-02-01
- Volume
- 17
- Issue
- 2
- Pages
- 133
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W7126397257
- Cited by
- 3
- References
- 62
- Citations by year
- 2026: 3
- FWCI
- 24.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- medical question answering, retrieval-augmented generation, hybrid retrieval, sparse retrieval, dense retrieval, BM25, hallucination mitigation, GPT-4o, BioGPT, faithfulness, medical knowledge base, PubMedQA, MedMCQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.



