Biomedical knowledge graph-optimized prompt generation for large language models
Open accessThe DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index; 8 of the 10 sources its claims cite were checked for support.
Abstract
MOTIVATION: Large language models (LLMs) are being adopted at an unprecedented rate, yet still face challenges in knowledge-intensive domains such as biomedicine. Solutions such as pretraining and domain-specific fine-tuning add substantial computational overhead, requiring further domain-expertise. Here, we introduce a token-optimized and robust Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework by leveraging a massive biomedical KG (SPOKE) with LLMs such as Llama-2-13b, GPT-3.5-Turbo, and GPT-4, to generate meaningful biomedical text rooted in established knowledge. RESULTS: Compared to the existing RAG technique for Knowledge Graphs, the proposed method utilizes minimal graph schema for context extraction and uses embedding methods for context pruning. This optimization in context extraction results in more than 50% reduction in token consumption without compromising the accuracy, making a cost-effective and robust RAG implementation on proprietary LLMs. KG-RAG consistently enhanced the performance of LLMs across diverse biomedical prompts by generating responses rooted in established knowledge, accompanied by accurate provenance and statistical evidence (if available) to substantiate the claims.
Overview
CoChat Summary
## TL;DR
This paper introduces KG-RAG, a token-optimized retrieval-augmented generation framework that uses the SPOKE biomedical knowledge graph to ground LLM outputs in established biomedical knowledge. Across benchmark tasks, it reduced token use by more than 50% versus prior KG-RAG approaches and improved Llama-2 performance on a challenging MCQ dataset by 71%.
## Key findings
- On a 100-question retrieval benchmark, KG-RAG achieved 97% retrieval accuracy, compared with 75% for Cypher-RAG and 61% for full-text index retrieval.
- Under prompt perturbation (lowercasing entity names), Cypher-RAG retrieval accuracy dropped to 0%, while KG-RAG maintained 97% and full-text index dropped slightly to 58% from 61%.
- KG-RAG used an average of 3,693 tokens per response, versus 8,006 for Cypher-RAG and 10,590 for full-text index, corresponding to 53.9% and 65.1% token reductions, respectively.
- On the MCQ dataset, KG-RAG increased Llama-2 accuracy from 0.31 ± 0.03 to 0.53 ± 0.03, described as approximately a 71% boost; on the same KG-RAG setup, GPT-3.5-Turbo outperformed GPT-4 on MCQ (0.79 ± 0.02 vs 0.74 ± 0.03, P < 0.0001).
## Methods
- Framework design: disease entity extraction from user prompts, disease-node matching in SPOKE, neighborhood/context retrieval from the graph, context pruning with sentence-transformer embeddings and cosine-similarity thresholds, then prompt assembly and LLM generation.
- Data/sample: SPOKE, a biomedical knowledge graph with 42 million nodes, 160 million edges, and 41 integrated biomedical databases; benchmark datasets included 311 True/False questions, 306 MCQs, and 100 retrieval-comparison questions, plus 165 validation questions for hyperparameter tuning.
- Analysis: compared KG-RAG with prompt-only, Cypher-RAG, and full-text index approaches using accuracy, Jaccard similarity, robustness to prompt perturbation, and token usage; LLMs tested were Llama-2-13b, GPT-3.5-Turbo, and GPT-4 with temperature set to 0.
## Limitations
- The paper states the approach is currently limited to biomedical questions centered on diseases, though it includes a proof of concept extending to more than 87,000 non-disease nodes; this is an explicit limitation.
- The paper says it was developed on SPOKE only and future work will extend it to other biomedical KGs and domains; this is an explicit limitation.
- The paper notes that context quality depends on the underlying graph and that SPOKE is “not entirely error free” or ready for clinical use; this is an explicit limitation.
- Evident from the text: the evaluation focuses on curated benchmark questions and retrieval tasks rather than a formal clinical validation of the KG-RAG system in real-world biomedical use.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
4 Valid · 2 Not valid · 0 Not checked · 50 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Thus, while SPOKE’s reliability has been demonstrated through its successful application in various biomedical contexts (Supplementary S1 Text; Himmelstein and Baranzini, 2015, Himmelstein et al. 2017, Nelson et al. 2019, Nelson et al. 2021a,b, Baranzini et al. 2022, Morris et al. 2023, Soman et al. 2023a,b, Tang et al. 2024)”
§ Biomedical knowledge graph-optimized prompt generation for large language models > 3 Results > 3.4 Performance on True/False and MCQ datasets
What the sources say
- Partly supportedcites Himmelstein and Baranzini, 2015 matched to “Heterogeneous Network Edge Prediction: A Data Integration Approach to Prioritize Disease-Associated Genes” (2015) sourceExperimental — accuracy not yet measured
The Discussion reports that this paper’s heterogeneous-network predictions successfully prioritized disease–gene associations across 29 diseases and helped identify three multiple-sclerosis susceptibility genes. That supports a successful biomedical application of a related network approach, but our copy does not identify the network as SPOKE or demonstrate SPOKE’s reliability across various contexts. (LLM, conf=0.83)
- Could not verifycites Himmelstein et al. 2017 matched to “Systematic integration of biomedical knowledge prioritizes drugs for repurposing” (2017) sourceExperimental — accuracy not yet measured
The Introduction and Results > Hetionet v1.0 describe Hetionet and its use in drug repurposing and prototype biomedical queries, not SPOKE. Searches for SPOKE and its reliability found no passage in our copy addressing whether SPOKE was successfully applied across biomedical contexts; page numbers were unavailable. (LLM, conf=0.76)
- Partly supportedcites Morris et al. 2023 matched to “The scalable precision medicine open knowledge engine (SPOKE): a massive knowledge graph of biomedical information” (2023) sourceExperimental — accuracy not yet measured
The Discussion reports that SPOKE has been used in several biomedical applications and that one approach predicted multiple sclerosis diagnoses with up to 83% accuracy. It does not establish SPOKE’s general reliability across those varied contexts; the documented success is specific to that prediction application. (LLM, conf=0.80)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Nevertheless, the “implicit representation” of knowledge in LLM has also been shown to generate non-factual information despite linguistically coherent answers (i.e. “hallucination”) as a response to the input prompt (Maynez et al. 2020, Raunak et al. 2021, Ji et al. 2023).”
§ Biomedical knowledge graph-optimized prompt generation for large language models > 1 Introduction
What the sources say
- Partly supportedcites Maynez et al. 2020 matched to “On Faithfulness and Factuality in Abstractive Summarization” (2020) sourceExperimental — accuracy not yet measured
Section 5.3, “Assessment of Factual Hallucinations” (page unavailable), reports that over 90% of BERTS2S hallucinations were erroneous, supporting the point that generated text can be non-factual. But the study concerns abstractive summaries of documents, not LLM answers to prompts, and it does not establish that an implicit representation of knowledge causes the errors; it says pretraining helps generate more factual summaries. (LLM, conf=0.86)
- Partly supportedcites Raunak et al. 2021 matched to “The Curious Case of Hallucinations in Neural Machine Translation” (2021) sourceExperimental — accuracy not yet measured
The Introduction describes NMT hallucinations as outputs decoupled from the source sequence despite being fluent in the target language, and the Conclusion reports experiments on when such hallucinations arise. This supports the claim’s general point about fluent but unsupported output. The paper studies neural machine translation, however, not LLM answers to prompts or an ‘implicit representation’ of knowledge. (LLM, conf=0.88)
- Not matchedcites Ji et al. 2023
This doesn’t read as a citation we can look up.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“To achieve this, we make use of the biomedical KG called SPOKE (Morris et al. 2023) that integrates more than 40 publicly available biomedical knowledge sources of separate domains where each source is centered around a biomedical concept, such as genes, proteins, drugs, compounds, diseases, and one or more of their known relationships.”
§ Biomedical knowledge graph-optimized prompt generation for large language models > 1 Introduction
What the sources say
- Supportedcites Morris et al. 2023 matched to “The scalable precision medicine open knowledge engine (SPOKE): a massive knowledge graph of biomedical information” (2023) source
The Introduction describes SPOKE as connecting 41 specialized databases across biomedical fields, using distinct node and relationship types. Table 1 in Section 2.1 lists sources for concepts including genes, proteins, compounds and diseases; the Results describe the relationships connecting them. Page numbers are not provided in our copy. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, we demonstrated that this method can be sensitive to how the prompt is formulated. Even slight perturbations to the prompt can lead to incorrect Cypher queries and subsequently impact downstream generative processes.”
§ Biomedical knowledge graph-optimized prompt generation for large language models > 2 Materials and methods > 2.8 Cypher-RAG
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Furthermore, KG-RAG enhanced the performance of proprietary GPT models, such as GPT-3.5 and GPT-4.”
§ Biomedical knowledge graph-optimized prompt generation for large language models > Abstract > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
25 more claims
Create a free account to see all 56 claims and every source verdict.
- Probably not a substantial claim. 7
- It may state two separate claims as one. 21
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Empirical evidence supporting the utility of LLMs in neurodegenerative disease characterization continues to emerge.”
From Review of Artificial Intelligence for Clinical Use in Alzheimer's Disease and Related Dementias (2026) · cites this paper as [21]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“provided neuropathologic differential diagnoses with high correlation to experts using human-curated clinical summaries.21”
From Review of Artificial Intelligence for Clinical Use in Alzheimer's Disease and Related Dementias (2026) · cites this paper as [21]
Experimental — accuracy not yet measured
Citations
Cites
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 1×
“This approach has led to the creation of models such as PubMedBERT (Lee et al. 2020, Gu et al. 2021) BlueBERT (Peng et al. 2019), SciBERT (Beltagy et al. 2019), ClinicalBERT (Huang et al. 2019), BioGPT (Luo et al. 2022), Med-PaLM (Singhal et al. 2022), and BioMedGPT (Luo et al. 2023).”
MethodSurvey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“Nevertheless, the “implicit representation” of knowledge in LLM has also been shown to generate non-factual information despite linguistically coherent answers (i.e. “hallucination”) as a response to the input prompt (Maynez et al. 2020, Raunak et al. 2021, Ji et al. 2023).”
BackgroundChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“This approach has led to the creation of models such as PubMedBERT (Lee et al. 2020, Gu et al. 2021) BlueBERT (Peng et al. 2019), SciBERT (Beltagy et al. 2019), ClinicalBERT (Huang et al. 2019), BioGPT (Luo et al. 2022), Med-PaLM (Singhal et al. 2022), and BioMedGPT (Luo et al. 2023).”
MethodRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
Piñero, J., Bravo, À., Queralt-Rosinach, N., Gutiérrez-Sacristán, A., Deu-Pons, J., Centeno, E., García-García, J., Sanz, F., & Furlong, L. I. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw943 · Cited by 2,680
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Research works printed but not linked: 1
- Kenton J-WC, Toutanova LK. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of naacL-HLT. 2019, 2.
Cited by
Retrieval augmented generation for large language models in healthcare: A systematic review
Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000877 · Cited by 267
A survey on knowledge-oriented retrieval-augmented generation
Cheng, M., Luo, Y., Ouyang, J., Liu, Q., Liu, H., Li, L., Yu, S., Zhang, B., Cao, J., Ma, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.10677 · Cited by 140
Zhao, X., Liu, S., Yang, S.-Y., & Miao, C. (2025) · Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3696410.3714782 · Cited by 116
Prompt Engineering in Clinical Practice: Tutorial for Clinicians
Liu, J., Liu, F., Wang, C., & Liu, S. (2025) · Journal of Medical Internet Research · doi:10.2196/72644 · Cited by 76
A comprehensive survey on integrating large language models with knowledge-based methods
Yang, W., Some, L., Bain, M., & Kang, B. H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113503 · Cited by 156
Xu, S., Yan, Z., Dai, C., & Wu, F. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1635381 · Cited by 56
A review on knowledge graphs for healthcare: Resources, applications, and promises
Cui, H., Lu, J., Xu, R., Wang, S.-Y., Ma, W., Yu, Y., Yu, S., Kan, X., Ling, C., Zhao, L., et al. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104861 · Cited by 127
Miao, Y., Zhao, Y., Luo, Y., Wang, H., & Wu, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/80557 · Cited by 38
Retrieval-Augmented Generation with Graphs (GraphRAG)
Han, H., Wang, Y., Shomer, H., Guo, K., Ding, J., Lei, Y., Halappanavar, M., Rossi, R. A., Mukherjee, S., Tang, X., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.00309 · Cited by 34
Bioinformatics and biomedical informatics with ChatGPT: Year one review
Wang, J.-G., Cheng, Z.-E., Yao, Q.-M., Liu, L., Xu, D., & Hu, G. (2024) · Quant. Biol. · doi:10.1002/qub2.67 · Cited by 3
Integrating Graph Retrieval-Augmented Generation With Large Language Models for Supplier Discovery
Li, Y., Ko, H., & Ameri, F. (2024) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4067389 · Cited by 18
Chen, G., Song, T., Wang, Q., Ma, Z., Hu, J., Li, Q., & Wu, C. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131230 · Cited by 28
A Survey on Unifying Large Language Models and Knowledge Graphs for Biomedicine and Healthcare
Xu, R., Jiang, P., Luo, L., Xiao, C., Cross, A., Pan, S., Sun, J., & Yang, C. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736556 · Cited by 38
MERMaid: Universal multimodal mining of chemical reactions from PDFs using vision-language models
Leong, S. X., Pablo‐García, S., Wong, B., & Aspuru‐Guzik, A. (2025) · Matter · doi:10.1016/j.matt.2025.102331 · Cited by 27
Leveraging LLMs and Knowledge Graphs to Design Secure Automation Systems
Hosseini, A. M., Kästner, W., & Sauter, T. (2025) · IEEE Open Journal of the Industrial Electronics Society · doi:10.1109/ojies.2025.3545811 · Cited by 13
Domain-Specific Question-Answering Systems: A Case Study of a Carbon Neutrality Knowledge Base
Liu, L., Zhou, Y., Ma, J., Zhang, Y., & He, L. (2025) · Sustainability · doi:10.3390/su17052192 · Cited by 25
A self-correcting Agentic Graph RAG for clinical decision support in hepatology
Hu, Y., Xuan, W., Zhou, Q., Li, Z., Li, Y., Hu, J., & Fang, F. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1716327 · Cited by 21
Mountantonakis, M., & Tzitzikas, Y. (2024) · Journal on Computing and Cultural Heritage · doi:10.1145/3708326 · Cited by 18
Combining LLMs and Knowledge Graphs to Reduce Hallucinations in Biomedical Question Answering
Pusch, L., & Conrad, T. (2025) · BioMedInformatics · doi:10.3390/biomedinformatics5040070 · Cited by 32
RARoK:Retrieval-Augmented Reasoning on Knowledge for Medical Question Answering
Zhan, B., Li, A., Yang, X., He, D., Duan, Y., & Yan, S. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822341 · Cited by 14
Construction and Application of Traffic Accident Knowledge Graph Based on LLM
Hou, Y., Shao, Y., Han, Z., & Ye, Z. (2025) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2025-01-7139 · Cited by 8
Reviewing clinical knowledge in medical large language models: Training and beyond
Li, Q. Y., Liu, H., Guo, C., Gao, C., Chen, D., Wang, M., Gao, F., van Harmelen, F., & Gu, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114215 · Cited by 20
Yan, W. P., Sun, H., Zhang, X., Liang, X., Zhu, S., & Jiang, Z. (2025) · Journal of Engineering Design · doi:10.1080/09544828.2025.2508124 · Cited by 9
He, J., Zhang, B., Rouhizadeh, H., Chen, Y., Yang, R., Lu, J. S., Chen, X., Liu, N., & Teodoro, D. (2026) · Artificial Intelligence Review · doi:10.21203/rs.3.rs-8330917/v1 · Cited by 33
Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graph
Zhang, Y., Chen, L., Li, S., Cao, N., Shi, Y., Ding, J., Qu, Z., Zhou, P., & Bai, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709187 · Cited by 21
Self-citationReview of Artificial Intelligence for Clinical Use in Alzheimer's Disease and Related Dementias
Breithaupt, A. G., Tang, A., Paolillo, E. W., Bibars, M., Johnson, E. C. B., Saloner, R., Possin, K. L., Windon, C. C., Hill-Jarrett, T. G., Giorgio, J., et al. (2026) · Seminars in Neurology · doi:10.1055/a-2744-9871 · Cited by 6
Boosting expertise and efficiency in LLM: A knowledge-enhanced framework for construction support
Yang, B., Xiao, H., Zeng, Z., Lai, S., Han, J., Tan, Y., & Ni, Y. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.09.029 · Cited by 4
Wang, S., Zhang, Y., Gao, Y., He, X., Deng, G., & Du, J. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf219 · Cited by 6
Yin, H., Parmar, P., Xu, D., Zhang, Y., Zheng, T., & Fu, W. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02672-4 · Cited by 21
Bridging data and discovery: a survey on knowledge graphs in AI for science
Ding, K., Zhu, Z., Tang, Y., Feng, K., Zhuang, X., Yang, Y., Yang, Y., Du, H., Ni, Z., Wang, S., et al. (2026) · National Science Review · doi:10.1093/nsr/nwag140 · Cited by 17
Gilani, K., Verket, M., Peters, C., Dumontier, M., Rocca, H.-P. B.-L., & Urovi, V. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110745 · Cited by 5
Large Language Models in Bio-Ontology Research: A Review
Manda, P. (2025) · Bioengineering · doi:10.3390/bioengineering12111260 · Cited by 15
Hossain, E., & Yousefi, N. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag219 · Cited by 10
Microbiome knowledge graph as a tool to understand bacteria-host associations
Pudavar, A. E., Baksi, K. D., Pokhrel, V., & Kuntal, B. K. (2025) · Archives of Microbiology · doi:10.1007/s00203-025-04413-0 · Cited by 4
Semantics-enabled biomedical literature analytics
Kilicoglu, H., Ensan, F., McInnes, B. T., & Wang, L. L. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104588 · Cited by 4
Wang, M., Shen, Y., Zhao, B., Zhou, X., Sun, L., & Liu, X. (2025) · Health Information Science and Systems · doi:10.1007/s13755-025-00379-x · Cited by 3
So, M. L., Park, S. J., Kim, D., Han, S., Koo, H. J., Kim, T., Shin, M.-G., & Lee, E. J. (2026) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.12.035 · Cited by 3
Desiderata for a biomedical knowledge network: opportunities, challenges and future directions
Wu, C., Liu, H., Flannick, J. A., Musen, M. A., Su, A. I., Hunter, L. E., Powers, T. M., & Wu, C. H. (2026) · Bioinformatics Advances · doi:10.1093/bioadv/vbag036 · Cited by 3
Trust-Aware KG-RAG: a framework for trustworthy question answering for engineering equipment
Shi, Q. W., Yang, Y., Di, J., Li, L., Liu, J., Hu, Y., Xiao, J., & Hu, Z. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104801 · Cited by 5
The Barancik award lecture: Multi-disciplinary research will be the key to stop, restore, and end MS
Baranzini, S. E. (2025) · Multiple Sclerosis Journal · doi:10.1177/13524585251314756 · Cited by 3
Self-citationWang, Z. P., Li, X., Matsumoto, N., Venkatesan, M., Chang, J.-H., Moran, J., Choi, H., Li, B., Meng, Y., Hernandez, M. E., et al. (2025) · BioData Mining · doi:10.1186/s13040-025-00466-5 · Cited by 13
Peng, C., He, E., & Xia, F. (2026) · IEEE Intelligent Systems · doi:10.1109/mis.2026.3650817 · Cited by 41
Retrieval Augmented Generation for Historical Newspapers
Tran, T. T., González-Gallardo, C.-E., & Doucet, A. (2024) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3677389.3702542 · Cited by 7
Yim, R. P., Silverman, A. L., Wang, S., & Rudrapatna, V. A. (2025) · BMJ Digital Health & AI · doi:10.1136/bmjdhai-2025-000014 · Cited by 2
Kumari, M., Chauhan, R., Jain, R., & Garg, P. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103902 · Cited by 6
Federated knowledge retrieval elevates large language model performance on biomedical benchmarks
Joy, J., & Su, A. I. (2026) · GigaScience · doi:10.1093/gigascience/giag007 · Cited by 3
RAG-based architectures for drug side effect retrieval using compact LLMs
Nygren, S., Erdogan, O., Avci, P., Daniels, A., Rassool, R., Beheshti, A., & Galeano, D. (2026) · Scientific Reports · doi:10.1038/s41598-026-41495-2 · Cited by 5
Medical multimodal large language models: A survey
Xiao, H., Hui, N., Xu, Y. D., Li, Z., & Peng, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104386 · Cited by 5
Exploring State-of-the-Art LLMs from BERT to XLNet: A study over Question Answering
Parra-Navarro, L. M., Batista, E. C. S., & Pacheco, M. A. C. (2024) · IEEE Latin American Conference on Computational Intelligence (LA-CCI) · doi:10.1109/la-cci62337.2024.10814828 · Cited by 5
Sawamura, S., Kohiyama, K., Takenaka, T., Sera, T., Inoue, T., & Nagai, T. (2025) · Cureus · doi:10.7759/cureus.79183 · Cited by 2
QAMT: An LLM-Based Framework for Quality-Assured Medical Time-Series Data Generation
Luo, Y., Zhang, Y., Xing, C., Ren, P., & Liu, X. (2025) · Sensors · doi:10.3390/s25175482 · Cited by 2
da Silva, R. P., & Pazin‐Filho, A. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03178-2 · Cited by 2
Fadillah, A. R., & Saputra, I. G. A. A. (2025) · Journal of Novel Engineering Science and Technology · doi:10.56741/jnest.v4i02.1049 · Cited by 1
Zhu, W., Zhang, P., Xia, W., et al. (2026) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s571156 · Cited by 1
Muasher-Kerwin, C., Hughes, M. C., Sanatizadeh, A., Mayer, J. F., & Alhoori, H. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-00879-z · Cited by 2
Chen, W., Zhou, Y., Yan, J., et al. (2025) · doi:10.1109/iccbdai66607.2025.11388981 · Cited by 1
KOSMOS: Ontology-Based Knowledge Graph Scaffolding for Medical Documentation Generation
Henry, R., & Gong, J. (2026) · Information · doi:10.3390/info17040355 · Cited by 2
A Pilot Empirical Study on When and How to Use Knowledge Graphs as Retrieval Augmented Generation
Yuan, X., Liu, Y., Di, S., Wu, S., Zheng, L., Meng, R., Chen, L., Zhou, X., & Yin, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0372-7_29 · Cited by 6
iS2C2: a cointelligent platform for mechanistic discovery of disease cellular crosstalk
Sheng, J., Ahn, J. Y., Yang, L., Wan, Z., Qi, S., Yu, X., Xu, Z., Cao, Y., Vasquez, M., Irfan, A., et al. (2026) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-026-02691-8 · Cited by 2
Zhao, Y., Miao, Y., Guo, R., Luo, Y., Wang, H., & Wu, Y. X. (2026) · Journal of Medical Internet Research · doi:10.2196/90046 · Cited by 2
Knowledge Graph-Enhanced Long-CoT for Complex Biomolecular Reasoning
Lyu, T., Zhuang, X., Ding, K., Cao, X., Liang, L., Zhao, W., Chen, H., & Zhang, Q. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818882 · Cited by 1
Kang, X., Ding, W., Matsumoto, K., Wang, L., Yu, H.-T., & Shi, X. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104702 · Cited by 1
UKRAG: A Unified Knowledge Graph to Enhance Retrieval Augmented Generation Performance
Alkouz, A., Al-Saleh, M. I., Alarabeyyat, A., & Bouchahma, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-82931-4_1 · Cited by 2
Knowledge Conceptualization Impacts RAG Efficacy
Jaldi, C. D., Saini, A., Ghiasi, E., Eziolise, O. D., & Shimizu, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-13109-6_15 · Cited by 4
Soleymani, S., Gravel, N., Kochut, K., & Kannan, N. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.23.649790 · Cited by 3
Automating Biomedical Knowledge Graph Construction For Context-Aware Scientific InferencePreprint
Zheng, Y., Liu, W., Zeng, B., Feng, Y., Du, X., Zhou, L., & Li, Y. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.14.699420 · Cited by 3
Yim, R. P., Silverman, A. L., Wang, S., et al. (2024) · medRxiv · doi:10.1101/2024.11.06.24316817
Prompt Engineering in Clinical Practice: A Comprehensive Guide for Clinicians (Preprint)Preprint
Liu, J., Wang, C., & Liu, S. (2025) · doi:10.2196/preprints.72644
Triple Play: A Framework for Enhanced Question Answering Reasoning with Kg, Llm, and RagPreprint
Hatem, S., Khoriba, A. P. G., Gad-Elrab, M. H., & Elhelw, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5277360
Bi-Encoder-Based Approach to Biomedical Document-Level Entity Recognition and Relation Extraction
Nie, P.-Y., Lin, M.-X., Ning, J.-Z., Yang, Z.-H., & Wang, L. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2025.3582790
Natural Language Querying of Biological Databases with Large Language ModelsPreprint
Makarov, V. A., Stroganov, O. V., Furlong, L. I., et al. (2025) · doi:10.31219/osf.io/94mex_v1
Med-KGMA: A novel AI-driven medical support system leveraging knowledge graphs and medical advisors
Varshney, S., Jain, B., Singh, P., Rani, D., & Mehra, S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110929
Jankov, A., Toshevska, M., & Gievska, S. (2025) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro65660.2025.11131859
Wu, R., Matsuno, H., & Ge, Q. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.02.673688
Biomedical Hypothesis Explainability with Graph-Based Context RetrievalPreprint
Tyagin, I., Valipour, S., Sikirzhytskaya, A., Shtutman, M. S., & Safro, I. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.17.683161 · Cited by 1
Gallingani, C., Miller, Z., Mandelli, M. L., Rosen, H. J., Ezzes, Z., Lin, M., Rodriguez, D., Grinberg, L. T., Spina, S., Seeley, W. W., et al. (2025) · medRxiv · doi:10.1101/2025.10.28.25338977 · Cited by 5
Sun, Y., Zhao, Y., Zhang, L., Luo, L., Zhou, Y., Zhang, Y., Han, X. M., & Zhou, M. (2025) · Journal of Dental Education · doi:10.1002/jdd.70101 · Cited by 2
Popova, E., Jacobs, M. D., Hofmann‐Apitius, M., & Babaiha, N. S. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100816 · Cited by 1
RehabKD With LLM-KG-LDACX: Orchestrating AI Models for Rehabilitation Knowledge Discovery
Zhang, W., Xing, Z., & Chen, S. (2025) · IEEE Access · doi:10.1109/access.2025.3645851
Research Directions for Ontology-Guided Domain-Specific Knowledge Graph Population Using LLMs
Saini, A., Jaldi, C. D., Ethier, J. G., & Shimizu, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-13109-6_10
Trust-Aware KG-RAG: A Framework for Trustworthy Question Answering for Engineering EquipmentPreprint
Shi, Q., Yang, Y., Di, J., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5955782
Generative Models for Clinical Entity Linking in SpanishPreprint
Zotova, E., Cuadros, M., & Rigau, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5976036
Empowering Large Language Models to Set Up Knowledge Retrieval Indexing via Self-Learning
Niu, S., Wang, M., Liang, X., li, Z., Zhang, S., Song, S., Wang, H., Yang, J., Xiong, F., & Xi, C. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3649907 · Cited by 1
Jin, L., Zhao, C., Han, D., Zuo, Z., Liang, Y., Liao, Y., & Tohti, T. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-025-02676-z · Cited by 2
Peng, Y., Hu, Y., Wang, L., Yang, H., Li, B., & Wang, Y. (2025) · International Conference on Digital Society and Intelligent Systems (DSInS) · doi:10.1109/dsins68311.2025.11329998 · Cited by 1
BiOmics: A Foundational Agent for Grounded and Autonomous Multi-omics InterpretationPreprint
Cao, L., Li, Y., Qin, H., Shang, Y., Zhang, Y., Jovanovic, B., Djokic, L., Xia, T., Hu, L., Hou, H., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.17.699830 · Cited by 3
Pei, L., Hu, J., Wanqing, Z., Wang, Q., & Fang, A. (2026) · doi:10.2196/preprints.92241
MetaKnogic-Alpha: A Hyper-Relational Knowledge Base for Grounded Metabolic ReasoningPreprint
Dang, P., Swaminathan, P., Guo, T., Wan, C., Cao, S., & Zhang, C. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.05.704050
Knowledge Graph-Enhanced Retrieval-Augmented Generation for Nutrigenetics
Filippis, G. M. D., Benfenati, D., Carlo, G. D., & Rinaldi, A. M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17216-7_8
EagleCare: Augmenting ICU Monitoring with LLMs and Knowledge Graph Reasoning
Guo, J., Gao, Y., Liu, Z., Rao, Y., Hu, J., & Wang, H. (2025) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg66886.2025.00024 · Cited by 2
Domain-Specific Knowledge Graphs in RAG-Enhanced Healthcare LLMS
Anuyah, S., Kaushik, M. M., Dai, H., Shiradkar, R., Arjan Durresi, A., & Chakraborty, S. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11400857
Wei, B., Xu, Y., He, Z., et al. (2025) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc60802.2025.11423447
Biomedical Knowledge Graph Embeddings with Transformers for Clinical Question Answering Systems
Alneamy, J. S. M., Saheb, N. M., AlTaei, M. H. A., & Baloul, F. M. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.01.067
Natural language querying of biological databases with large language models
Makarov, V. A., Stroganov, O. V., Furlong, L. I., Evarts, B., van den Biggelaar, L., Goulas, A., Stolte, E., Marren, D., & Greiffenberg, L. (2026) · Drug Discovery Today · doi:10.1016/j.drudis.2026.104654 · Cited by 1
Comparative Evaluation of Large Language Models in Clinical Diagnostics for Real-World Medical Cases
Młodawski, M., Deniziak, S. R., Płaza, M., Kwiatkowska, A., Jaszczyk, D., Gajda, A., & Twardowski, K. (2026) · Applied Sciences · doi:10.3390/app16073499 · Cited by 1
Generative models for clinical entity linking in Spanish
Zotova, E., Cuadros, M., & Rigau, G. (2026) · Array · doi:10.1016/j.array.2026.100805
Happi, B. G. H. (2024) · theses.fr (ABES) · doi:10.70675/20022d2az1be9z4bd9za6e1zb410bde66ce1 · Cited by 1
INFERMed: A PK/PD-Aware Retrieval-Augmented System for Explainable Drug-Drug Interaction Analysis
Mishra, P., Ganzha, M., Plewczyński, D., & Paprzycki, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-23241-0_9
Biomed-R 2 : Joint Diversity Retrieval and Evidence Reasoning for Biomedical Question Answering
Guan, H., Li, C., Zhang, W., Lian, Y., Guo, J., & Lü, W. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463381 · Cited by 1
Fu, Y., Lyu, T., Chen, Y., & Lv, Z. (2026) · ENGINEERING Management · doi:10.1007/s42524-026-4237-0 · Cited by 2
Metadata
- Authors
- Soman, K., Rose, P. W., Morris, J. H., Akbas, R. E., Smith, B., Peetoom, B., Villouta-Reyes, C., Cerono, G., Shi, Y., Rizk-Jackson, A., Israni, S., Nelson, C. A., Huang, S., & Baranzini, S. E.
- Affiliations
- University of California, San Francisco (US)
- San Diego Supercomputer Center (US)
- Institute for Systems Biology (US)
- Bakar Computational Health Sciences Institute, University of California , San Francisco, CA 94158,
- Mate Bioservices, Inc. Swallowtail Ct. , Brisbane, CA 94005,
- Corresponding author
- Baranzini, Sergio E
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-09
- Volume
- 40
- Issue
- 9
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC11441322
- OpenAlex
- W4402567773
- Cited by
- 135
- References
- 35
- Citations by year
- 2026: 61 · 2025: 63 · 2024: 9
- FWCI
- 10.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- PubMed Central (submittedVersion, cc-by)
- Funders
- National Science Foundation (NSF_2033569)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- knowledge graph retrieval-augmented generation, prompt generation, large language models, biomedical knowledge graphs, context extraction, Llama 2, GPT-4, GPT-3.5-turbo
- MeSH terms
- Algorithms, Humans, Natural Language Processing, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.