The DOI matches its Crossref record; 74 of the 91 references it cites were found in a registry or an index, and 10 are not checked yet; 35 of the 47 sources its claims cite were checked for support.
Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework
Open accessAbstract
Evaluation was conducted on PubMedQA, MedMCQA, and MedQA-US using hybrid retrieval combining BM25 and MedCPT with GPT-4o and BioGPT; the hybrid retriever achieved 92.14% recall, 74.36% precision, and 82.30% F1, and GPT-4o with hybrid retrieval reached 89.4% faithfulness on PubMedQA.
Verified 2026-10-10 · 10 references not checked yet
Overview
CoChat Summary
## TL;DR
This paper proposes a modular retrieval-augmented generation (RAG) framework for medical question answering that combines BM25 and MedCPT retrieval with GPT-4o and BioGPT generation. On three healthcare QA datasets, the hybrid retriever outperformed single retrievers, and GPT-4o generally produced more faithful and better-balanced answers than BioGPT.
## Key findings
- The proposed hybrid retriever achieved 92.14% context recall, 74.36% context precision, and 82.30% F1-score, outperforming BM25 (82.76% recall, 67.77% precision, 74.52% F1) and MedCPT (73.31% recall, 48.75% precision, 58.56% F1).
- On PubMedQA, GPT-4o with hybrid retrieval reached 89.4% faithfulness, 82.7% answer relevancy, 93.1% context precision, 80.6% context recall, and 88.0% BERTScore F1.
- On MedMCQA, GPT-4o achieved 78.1% answer relevancy and 77.9% context recall; on MedQA-US, it reached 34.7% faithfulness and 77.3% context recall.
- BioGPT had very high context precision on PubMedQA (98.4%) but much lower faithfulness (32.3%), suggesting strong lexical overlap without consistently reliable grounding.
## Methods
- The study used a modular RAG pipeline with four phases: data preparation and indexing, hybrid retrieval, response generation, and evaluation.
- It evaluated three healthcare QA datasets: PubMedQA, MedMCQA, and MedQA-US; the text also says retrieval evaluation used a 15,000-question corpus, and generation evaluation was done on 3,000 questions sampled evenly from the three datasets.
- Retrieval compared BM25, MedCPT, and a hybrid late-fusion approach that merged both retrievers; generation compared GPT-4o (few-shot prompting, temperature = 0, max_tokens = 512) and BioGPT (beam search).
- Evaluation used context precision, context recall, F1, Average Precision, plus RAGAS and BERTScore for generation quality.
## Limitations
- The paper explicitly states that it does not perform disease-specific or organ-level stratified analysis, so performance differences across subdomains such as cardiology, nephrology, and oncology were not separately examined.
- It also states that future work could extend to subdomain-aware evaluation; this makes the current evaluation broad rather than specialty-specific.
- Evident from the text: the framework was tested on only three datasets, and the reported generation evaluation was sampled from 3,000 questions, so the results may not cover all medical QA settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
19 Valid · 7 Not valid · 10 Not checked · 96 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“By incorporating external information retrieval, RAG enhances generative models with improved accuracy and credibility for knowledge-intensive tasks.”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 1. Introduction
What the sources say
- Supportedcites [8] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., and Wang, H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv. source
The Introduction (page not specified in our copy) says RAG enhances LLMs by retrieving information from external knowledge bases and reduces factually incorrect content. It identifies knowledge-intensive tasks as a setting where LLMs face limitations; the Overview likewise describes retrieval helping them produce well-informed answers. These passages support the claim’s accuracy and credibility paraphrase. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Most existing Modular RAG systems in healthcare, such as Self-BioRAG [16] and Clinfo.ai[24], rely solely on either sparse (BM25) or dense (MedCPT [25], embedding-based) retrieval methods.”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 2. Related Works
What the sources say
- Supportedcites [16] Jeong, 2024, Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models, Bioinformatics source
The citation identifies Self-BioRAG as an example. Section 3.2, “Biomedical retriever,” says Self-BioRAG uses MedCPT to retrieve evidence, followed by reranking. This supports the retrieval-method attribution for Self-BioRAG; this citation is not offered as evidence for Clinfo.ai or for the prevalence claim about most systems. (LLM, conf=0.90)
- Not checked yetcites [24] Lozano, A., Fleming, S.L., Chiang, C.C., and Shah, N. (2024, January 3–7). Clinfo.ai: An open-source retrieval-augmented large language model system for answering medical questions using scientific literature. Proceedings of the Pacific Symposium on Biocomputing 2024, Honolulu, HI, USA. source
Checked before — its earlier verdict is held back until a new check runs.
- Supportedcites [25] Jin, 2023, MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval, Bioinformatics source
The citation is attached specifically to MedCPT, identifying it as a dense, embedding-based retrieval method. Section 2.2 describes its query and document encoders and nearest-neighbor search; the Introduction explains that dense retrievers encode queries and documents as vectors. The citation does not attach to the broader assertion about most Modular RAG systems. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, current RAG models face challenges related to frequent knowledge updates [2,11,21,22].”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 1. Introduction
What the sources say
- Could not verifycites [2] Yang, 2023, Large language models in health care: Development, applications, and challenges, Health Care Sci. sourceExperimental — accuracy not yet measured
The 'Questionable credibility and accuracy of information' section discusses inaccurate LLM outputs, but not retrieval-augmented generation or difficulties with frequent knowledge updates. Searches for RAG, retrieval-augmented generation, and updated or outdated knowledge found no passage addressing the attributed claim; no page number is available in our copy. (LLM, conf=0.75)
- Partly supportedcites [11] Li, 2024, A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges, Vicinagearth sourceExperimental — accuracy not yet measured
Section 3.3 describes RAG as retrieving information from external knowledge sources and says such approaches still fall short for more profound knowledge updates. It also discusses the difficulty of keeping LLM knowledge current, but does not specifically say that frequent updates pose a challenge for RAG models. (LLM, conf=0.82)
- Partly supportedcites [21] Huang, J., Wang, M., Cui, Y., Liu, J., Chen, L., Wang, T., and Wu, J. (2024). Layered Query Retrieval: An Adaptive Framework for Retrieval-Augmented Generation in Complex Question Answering for Large Language Models. Appl. Sci., 14. sourceExperimental — accuracy not yet measured
Our copy does state that current RAG approaches carry unresolved limitations, but the challenges it enumerates are query-complexity handling, efficiency vs. accuracy, computational cost of iterative retrieval, and inconsistent/contradictory information when merging retrieved documents (Related Work; Introduction) — never the freshness or updating of knowledge. On knowledge currency the paper points the other way: it attributes the 'up-to-date knowledge'/knowledge-cutoff problem to pre-trained LLMs relying on static data and presents RAG as mitigating it (Introduction). So the 'current RAG models face challenges' part is supported, while the specific qualifier 'related to frequent knowledge updates' is not attributable to this source. (LLM, conf=0.68)
- Partly supportedcites [22] Xu, K., Zhang, K., Li, J., Huang, W., and Wang, Y. (2024). CRP-RAG: A Retrieval-Augmented Generation Framework for Supporting Complex Logical Reasoning and Knowledge Planning. Electronics, 14. sourceExperimental — accuracy not yet measured
Our copy does state that current RAG research faces challenges — the Introduction ('However, current RAG research still faces two issues') and Section 1 ('the RAG framework still faces two major challenges') — but the challenges it identifies are irrelevant-knowledge interference with reasoning, linear reasoning structures that block complex thought transformations, and the decoupling of knowledge retrieval/planning from the reasoning process, not frequent knowledge updates. No passage in our copy (searched: knowledge updates, outdated/temporal knowledge, knowledge-base refresh, dynamic/static knowledge modeling) attributes a knowledge-update difficulty to current RAG models; the closest text, Section 6.3, concerns CRP-RAG's own lack of 'autonomous evolution and coordinated updates based on environmental changes', i.e., a limitation of the proposed framework rather than of current RAG generally. Thus the generic frame of the claim is supported while the specific qualifier ('related to frequent knowledge updates') is not. (LLM, conf=0.62)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This pattern, observed in approximately 25% of cases, reflects instruction-following limitations where RAG systems struggle to align response style with the specific demands of the question type [52,60].”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 4.1. Error Analysis
What the sources say
- Partly supportedcites [52] Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2024, January 17–22). RAGAS: Automated Evaluation of Retrieval Augmented Generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, St. Julian’s, Malta. sourceExperimental — accuracy not yet measured
The Answer relevance section (page unavailable) says the metric assesses whether an answer appropriately addresses and aligns with the initial question or instruction. It does not characterize mismatches as instruction-following limitations or discuss response style and the demands of different question types. The cited paper also does not establish the approximately 25% figure, though that may describe the citing authors’ own observations. (LLM, conf=0.84)
- Partly supportedcites [60] Schmidgall, 2024, Evaluation and mitigation of cognitive biases in medical language models, npj Digit. Med. sourceExperimental — accuracy not yet measured
Methods > Automated evaluation (page unavailable) reports that two medical language models frequently ignored an instruction to answer with only a single letter, supporting the general point about instruction-following and response format. It does not establish a roughly 25% rate or examine RAG systems matching response style to different question types. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“At the retrieval stage, three primary error types were observed: context recall error [52], faithfulness error [52], and answer relevance mismatch [52].”
§ Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework > 4.1. Error Analysis
What the sources say
- Partly supportedcites [52] Es, S., James, J., Espinosa-Anke, L., and Schockaert, S. (2024, January 17–22). RAGAS: Automated Evaluation of Retrieval Augmented Generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, St. Julian’s, Malta. sourceExperimental — accuracy not yet measured
Section 3, “Evaluation Strategies” (page unavailable), identifies three quality aspects: faithfulness, answer relevance, and context relevance—not context recall. It defines faithfulness and answer relevance in terms of the generated answer, so the source does not support describing all three as error types observed at the retrieval stage. (LLM, conf=0.88)
plainn = 30- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
67 more claims · 7 with their cited sources checked
Create a free account to see all 132 claims and to verify the 5 references still open.
- It may state two separate claims as one. 38
- Probably not a substantial claim. 40
- Borderline: one of the tests for a substantial claim is unclear. 1
Citations
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Probable Inference, the Law of Succession, and Statistical Inference
Wilson, E. B. (1927) · Journal of the American Statistical Association · doi:10.1080/01621459.1927.10502953 · Cited by 4,476
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D. S., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1705.03551 · Cited by 1,650
National Standards for Diabetes Self-Management Education
Funnell, M. M., Brown, T. L., Childs, B. P., Haas, L. B., Hosey, G. M., Jensen, B., Maryniuk, M., Peyrot, M., Piette, J. D., Reader, D., et al. (2009) · Diabetes Care · doi:10.2337/dc09-s087 · Cited by 1,350
Cited by
Monika, W., Dewi, D. A., Nasution, A. H., Onan, A., & Murakami, Y. (2026) · Information · doi:10.3390/info17070660 · Cited by 1
“Recent studies include hybrid medical RAG, knowledge-graph medical QA, clinical decision support, electronic health records, and evidence-based hallucination reduction [28,32,52,61,63,90,107,109].”
Development and Multireader Evaluation of Radiological RAG-System
Erizhokov, R. A., Gordeev, A. E., Sakharova, P. A., Yafarova, A. A., Varyukhina, M. D., Blokhin, I. A., Omelyanskaya, O. V., Vladzymyrskyy, A. V., & Vasilev, Y. A. (2026) · Data · doi:10.3390/data11060143 · Cited by 1
“This number was selected empirically as an optimal trade-off between coverage completeness and computational efficiency: a smaller number reduced contextual completeness, whereas a larger number increased noise and response time [27].”
Jamil, M., Kavak, A., Omurca, S. İ., & Fotouhi, H. (2026) · Diagnostics · doi:10.3390/diagnostics16193142
Metadata
- Authors
- Aljohani, B., & Alsanoosy, T.
- Affiliations
- Taibah University (SA)
- Corresponding author
- Alsanoosy, Tawfeeq
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2026-02-01
- Volume
- 17
- Issue
- 2
- Pages
- 133
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W7126397257
- Cited by
- 3
- References
- 62
- Citations by year
- 2026: 3
- FWCI
- 24.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- medical question answering, retrieval-augmented generation, hybrid retrieval, sparse retrieval, dense retrieval, BM25, hallucination mitigation, GPT-4o, BioGPT, faithfulness, medical knowledge base, PubMedQA, MedMCQA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework. https://app.cochat.ai/papers/10.3390/info17020133
- The paper
- Aljohani, B., & Alsanoosy, T. (2026). Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework. Information. https://doi.org/10.3390/info17020133
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.



