CoChat
Sign Up
Cited by 132Artificial Intelligence in Healthcare and EducationTopic ModelingInformation Retrieval and Search Behavior

Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models

Open access
Jeong, Minbyul, Sohn, Jiwoong, Sung, Mujeen, Kang, Jaewoo·2024·Bioinformatics
DOI 10.1093/bioinformatics/btae238PMC PMC11211826
Open this paper in CoChat — free account

Abstract

Summary: Recent proprietary large language models (LLMs), such as GPT-4, have achieved a milestone in tackling diverse challenges in the biomedical domain, ranging from multiple-choice questions to long-form generations. To address challenges that still cannot be handled with the encoded knowledge of LLMs, various retrieval-augmented generation (RAG) methods have been developed by searching documents from the knowledge corpus and appending them unconditionally or selectively to the input of LLMs for generation. However, when applying existing methods to different domain-specific problems, poor generalization becomes apparent, leading to fetching incorrect documents or making inaccurate judgments. In this paper, we introduce Self-BioRAG, a framework reliable for biomedical text that specializes in generating explanations, retrieving domain-specific documents, and self-reflecting generated responses. We utilize 84k filtered biomedical instruction sets to train Self-BioRAG that can assess its generated explanations with customized reflective tokens. Our work proves that domain-specific components, such as a retriever, domain-related document corpus, and instruction sets are necessary for adhering to domain-related instructions. Using three major medical question-answering benchmark datasets, experimental results of Self-BioRAG demonstrate significant performance gains by achieving a 7.2% absolute improvement on average over the state-of-the-art open-foundation model with a parameter size of 7B or less. Similarly, Self-BioRAG outperforms RAG by 8% Rouge-1 score in generating more proficient answers on two long-form question-answering benchmarks on average. Overall, we analyze that Self-BioRAG finds the clues in the question, retrieves relevant documents if needed, and understands how to answer with information from retrieved documents and encoded knowledge as a medical expert does. We release our data and code for training our framework components and model weights (7B and 13B) to enhance capabilities in biomedical and clinical domains. Availability and implementation: Self-BioRAG is available at https://github.com/dmis-lab/self-biorag.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btae238

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Extensive literature consistently reports that domain-specific models such as BioGPT are prone to hallucination or generating content beyond the retrieved evidence unless explicitly fine-tuned for retrieval-grounded generation.

    “Moreover, extensive literature consistently reports that while domain-specific models such as BioGPT excel at producing semantically rich and lexically appropriate text, they are prone to hallucination or generating content beyond the retrieved evidence unless explicitly fine-tuned for retrieval-grounded generation [16,24,29,32]”

    From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [16]

    The Introduction (page not provided) describes hallucination as a problem for LLMs and says retrieval-augmented generation supplies supporting facts. It does not make that finding specifically about BioGPT, establish that domain-specific models produce text beyond retrieved evidence, or say retrieval-grounded fine-tuning is necessary. The Results and analysis section (page not provided) explicitly says its reported scores cannot measure how much hallucination occurs. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Most existing Modular RAG systems in healthcare rely solely on either sparse (BM25) or dense (MedCPT, embedding-based) retrieval methods.

    “Most existing Modular RAG systems in healthcare, such as Self-BioRAG [16] and Clinfo.ai[24], rely solely on either sparse (BM25) or dense (MedCPT [25], embedding-based) retrieval methods.”

    From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [16]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 37

  1. Abacha AB, Yassine M, Mark S et al Bridging the gap between consumers’ medication questions and trusted answers. In: MedInfo. 2019. 31437878
  2. Asai A, Zeqiu W, Yizhong W. et al Self-rag: learning to retrieve, generate, and critique through self-reflection. arXiv, arXiv:2310.11511, 2023, preprint: not peer reviewed. · arXiv
  3. Bajaj P, Daniel C, Nick C. et al Ms marco: a human generated machine reading comprehension dataset. arXiv, arXiv:1611.09268, 2016, preprint: not peer reviewed. · arXiv
  4. Cao M, Yue D, Jackie C, Kit C. Hallucinated but factual! inspecting the factuality of hallucinations in abstractive summarization. In: Association for Computational Linguistics, Dublin, Ireland. 2022.
  5. Chen Z Alenjandro H, Cano AR et al Meditron-70b: Scaling medical pretraining for large language models. arXiv, arXiv:2311.16079, 2023, preprint: not peer reviewed. · arXiv
  6. Christiano PF Jan L, Brown TB et al Deep reinforcement learning from human preferences. In: Advances in Neural Information Processing Systems, Long Beach, CA, USA. 2017.
  7. Chung HW Le H, Shayne L et al Scaling instruction-finetuned language models. arXiv, arXiv:2210.11416, 2022, preprint: not peer reviewed. · arXiv
  8. Fang Y Xiaozhuan L, Ningyu Z et al Mol-instructions: a large-scale biomolecular instruction dataset for large language models. arXiv, arXiv:2306.08018, 2023, preprint: not peer reviewed. · arXiv
  9. Guu K Kenton L, Zora T et al Retrieval augmented language model pre-training. In: International Conference On Machine Learning. 2020.
  10. Han T Adams LC, Papaioannou JM et al Medalpaca—an open-source collection of medical conversational ai models and training data. arXiv, arXiv:2304.08247, 2023, preprint: not peer reviewed. · arXiv
  11. Hendrycks D Collin B, Steven B et al Measuring massive multitask language understanding. arXiv, arXiv:2009.03300, 2020, preprint: not peer reviewed. · arXiv
  12. Izacard G Patrick L, Maria L et al Few-shot learning with retrieval augmented language models. arXiv, arXiv:2208.03299, 2022a, preprint: not peer reviewed. · arXiv
  13. Izacard G Mathilde C, Lucas H et al Unsupervised dense information retrieval with contrastive learning. In: Transactions on Machine Learning Research. 2022b.
  14. Ji Z Nayeon L, Rita F et al Survey of hallucination in natural language generation. In: ACM Computing Surveys. 2023.
  15. Jiang Z Frank X, Luyu G et al Active retrieval augmented generation. arXiv, arXiv:2305.06983, 2023, preprint: not peer reviewed. · arXiv
  16. Jin Q, Kim W, Chen Q et al Medcpt: contrastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval. Bioinformatics 2023; 39: btad651. 37930897 PMC10627406
  17. Kang M Seanie L, Jinheon B et al Knowledge-augmented reasoning distillation for small language models in knowledge-intensive tasks. arXiv, arXiv:2305.18395, 2023, preprint: not peer reviewed. · arXiv
  18. Kwon W Zhuohan L, Siyuan Z et al Efficient memory management for large language model serving with pagedattention. In: Symposium on Operating Systems Principles. 2023.
  19. Lewis P Ethan P, Aleksandra P et al Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems. 2020.
  20. Li B Tianxin M, Xiaoming S et al Meddm: Llm-executable clinical guidance tree for clinical decision-making. arXiv, arXiv:2312.02441, 2023, preprint: not peer reviewed. · arXiv
  21. Mao Y Pengcheng H, Xiaodong L et al Generation-augmented retrieval for open-domain question answering. In: Association for Computational Linguistics and International Joint Conference on Natural Language Processing. 2021.
  22. Nori H Nicholas K, Scott MM et al Capabilities of gpt-4 on medical challenge problems. arXiv, arXiv:2303.13375, 2023, preprint: not peer reviewed. · arXiv
  23. Ouyang L Jeff W, Xu J et al Training language models to follow instructions with human feedback. In: Advances in Neural Information Processing Systems. 2022.
  24. Pal A Logesh Kumar U, Malaikannan S. Medmcqa: a large-scale multi-subject multi-choice dataset for medical domain question answering. In: Conference on Health, Inference, and Learning. 2022.
  25. Schulman J Filip W, Prafulla D et al Proximal policy optimization algorithms. arXiv, arXiv:1707.06347, 2017, preprint: not peer reviewed. · arXiv
  26. Shao Z Yeyun G, Yelong S et al Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. arXiv, arXiv:2305.15294, 2023, preprint: not peer reviewed. · arXiv
  27. Singhal K Shekoofeh A, Tao T et al Large language models encode clinical knowledge. arXiv, arXiv:2212.13138, 2022, preprint: not peer reviewed. · arXiv
  28. Taylor R Marcin K, Guillem C et al Galactica: A large language model for science. arXiv, arXiv:2211.09085, 2022, preprint: not peer reviewed. · arXiv
  29. Touvron H Louis M, Kevin S et al Llama 2: Open foundation and fine-tuned chat models. arXiv, arXiv:2307.09288, 2023, preprint: not peer reviewed. · arXiv
  30. Wang Y Yeganeh K, Swaroop M et al Self-instruct: aligning language model with self generated instructions. Toronto, Canada. arXiv, arXiv:2212.10560, 2022, preprint: not peer reviewed. · arXiv
  31. Wang Y Xueguang M, Wenhu C Augmenting black-box llms with medical textbooks for clinical question answering. arXiv, arXiv:2309.02233, 2023, preprint: not peer reviewed. · arXiv
  32. Wang Z Patrick N, Xiaofei M et al Multi-passage Bert: A globally normalized Bert model for open-domain question answering. In: Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing, Hong Kong, China. 2019.
  33. Wei J Xuezhi W, Dale S et al Chain-of-thought prompting elicits reasoning in large language models. In: Advances in Neural Information Processing Systems. 2022.
  34. Wu C Weixiong L, Xiaoman Z et al Pmc-llama: Further finetuning llama on medical papers. arXiv, arXiv:2304.14454, 2023a, preprint: not peer reviewed. · arXiv
  35. Wu Z Yushi H, Weijia S et al Fine-grained human feedback gives better rewards for language model training. arXiv, arXiv:2306.01693, 2023b, preprint: not peer reviewed. · arXiv
  36. Zhang T Varsha K, Felix W et al Bertscore: evaluating text generation with Bert. In: International Conference on Learning Representations. 2019.
  37. Zhang X Chenxin T, Xianjun Y et al Alpacare: instruction-tuned large language models for medical application. arXiv, arXiv:2310.14558, 2023, preprint: not peer reviewed. · arXiv

Cited by

Sorted by citations

Authors
Jeong, M., Sohn, J., Sung, M., & Kang, J.
Affiliations
  • Korea University (KR)
  • Kyung Hee University (KR)
Corresponding author
  • Sung, Mujeen
  • Kang, Jaewoo
ORCID
Venue
Bioinformatics
Publisher
Oxford University Press (OUP)
Published
2024-06-28
Volume
40
Issue
Supplement_1
Pages
i119-i129
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
DOI
10.1093/bioinformatics/btae238
PMC
PMC11211826
OpenAlex
W4400128031
Cited by
132
References
0
Citations by year
2026: 62 · 2025: 53 · 2024: 17
FWCI
12.34
Type
journal article
Language
en
Open access
Licence
Locations
  • Bioinformatics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Research Foundation of Korea (2023R1A2C3004176, 2022R1C1C1008074)
  • Ministry of Health & Welfare, Republic of Korea (HR20C0021(3), HR22C1302)
  • Ministry of Science and ICT (RS-2023–00262002, RS-2022-00155911)
  • Kyung Hee University
  • Institute of Information & Communications Technology Planning & Evaluation
  • MSIT (IITP-2024–2020-0-01819)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
retrieval-augmented generation, large language models, medical question answering, biomedical text, self-reflection, explanation generation, document retrieval, long-form question answering, medical reasoning
MeSH terms
Humans, Natural Language Processing, Information Storage and Retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review