GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information
Open accessAbstract
Motivation: While large language models (LLMs) have been successfully applied to various tasks, they still face challenges with hallucinations. Augmenting LLMs with domain-specific tools such as database utilities can facilitate easier and more precise access to specialized knowledge. In this article, we present GeneGPT, a novel method for teaching LLMs to use the Web APIs of the National Center for Biotechnology Information (NCBI) for answering genomics questions. Specifically, we prompt Codex to solve the GeneTuring tests with NCBI Web APIs by in-context learning and an augmented decoding algorithm that can detect and execute API calls. Results: Experimental results show that GeneGPT achieves state-of-the-art performance on eight tasks in the GeneTuring benchmark with an average score of 0.83, largely surpassing retrieval-augmented LLMs such as the new Bing (0.44), biomedical LLMs such as BioMedLM (0.08) and BioGPT (0.04), as well as GPT-3 (0.16) and ChatGPT (0.12). Our further analyses suggest that: First, API demonstrations have good cross-task generalizability and are more useful than documentations for in-context learning; second, GeneGPT can generalize to longer chains of API calls and answer multi-hop questions in GeneHop, a novel dataset introduced in this work; finally, different types of errors are enriched in different tasks, providing valuable insights for future improvements. Availability and implementation: The GeneGPT code and data are publicly available at https://github.com/ncbi/GeneGPT.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
DbQA questions were designed to cover a wide array of data sources, such that a model or agent would not be able to answer them all with the use of a single API.
“DbQA questions were designed to cover a wide array of data sources, such that a model or agent would not be able to answer them all with the use of a single API [20].”
From LAB-Bench: Measuring Capabilities of Language Models for Biology Research (2024) · cites this paper as [20]
The GeneTuring and RQ2: Multi-hop QA on GeneHop sections describe GeneTuring and GeneHop questions, not DbQA. Searches for DbQA and for questions spanning data sources returned no passage about DbQA’s design or whether a single API could answer all its questions. Page numbers are unavailable in our copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 3× · see citing works
“However, such auto-regressive LLMs are still susceptible to hallucinations and generate erroneous content (Ji et al., 2023).”
BackgroundScaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works
“Large language models: Recent studies have shown that scaling pre-trained LMs leads to performance improvement and potentially emergent abilities on various NLP tasks (Brown et al., 2020; Kaplan et al., 2020; Wei et al., 2022a; Chowdhery et al., 2022; OpenAI, 2023).”
BackgroundOpportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad493 · Cited by 359 · see citing works
Self-citationCan large language models reason about medical questions?
Liévin, V., Hother, C. E., Motzfeldt, A. G., & Winther, O. (2024) · Patterns · doi:10.1016/j.patter.2024.100943 · Cited by 343 · cited 1× · see citing works
“They also achieve state-of-the-art performance on many domain-specific tasks like biomedical question answering (Singhal et al., 2022; Liévin et al., 2022; Nori et al., 2023).”
Background
Flagged first, then by citations
Research works printed but not linked: 27
- Altschul SF, Gish W, Miller W, Myers EW, and Lipman DJ (1990). Basic local alignment search tool. Journal of molecular biology, 215(3):403-410.
- Borgeaud S, Mensch A, Hoffmann J, Cai T, Rutherford E, Millican K, Van Den Driessche GB, Lespiau JB, Damoc B, Clark A, et al. (2022). Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206-2240.
- PMLR.
- Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
- Chen M, Tworek J, Jun H, Yuan Q, Pinto HPdO, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G, et al. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. · arXiv
- Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S, et al. (2022). Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. · arXiv
- Ely JW, Osheroff JA, Chambliss ML, Ebell MH, and Rosenbaum ME (2005). Answering physicians' clinical questions: obstacles and potential solutions. Journal of the American Medical Informatics Association, 12(2):217-224.
- Gao L, Madaan A, Zhou S, Alon U, Liu P, Yang Y, Callan J, and Neubig G (2022). Pal: Program-aided language models. arXiv preprint arXiv:2211.10435. · arXiv
- Guu K, Lee K, Tung Z, Pasupat P, and Chang M (2020). Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929-3938. PMLR.
- Hou W and Ji Z (2023). Geneturing tests gpt models in genomics. bioRxiv, pages 2023-03.
- Jin Q, Leaman R, and Lu Z (2023a). Retrieve, summarize, and verify: How will chatgpt impact information seeking from the medical literature? Journal of the American Society of Nephrology, pages 10-1681.
- Jin Q, Wang Z, Floudas CS, Sun J, and Lu Z (2023b). Matching patients to clinical trials with large language models. arXiv preprint arXiv:2307.15051. · arXiv
- Jin Q, Yuan Z, Xiong G, Yu Q, Ying H, Tan C, Chen M, Huang S, Liu X, and Yu S (2022). Biomedical question answering: a survey of approaches and challenges. ACM Computing Surveys (CSUR), 55(2):1-36.
- Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih Wt, Rocktäschel T, et al. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459-9474.
- Mialon G, Dessì R, Lomeli M, Nalmpantis C, Pasunuru R, Raileanu R, Rozière B, Schick T, Dwivedi-Yu J, Celikyilmaz A, et al. (2023). Augmented language models: a survey. arXiv preprint arXiv:2302.07842. · arXiv
- Nori H, King N, McKinney SM, Carignan D, and Horvitz E (2023). Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375. · arXiv
- OpenAI (2023). GPT-4 technical report. CoRR, abs/2303.08774. · arXiv
- Parisi A, Zhao Y, and Fiedel N (2022). Talm: Tool augmented language models. arXiv preprint arXiv:2205.12255. · arXiv
- Qin Y, Hu S, Lin Y, Chen W, Ding N, Cui G, Zeng Z, Huang Y, Xiao C, Han C, et al. (2023). Tool learning with foundation models. arXiv preprint arXiv:2304.08354. · arXiv
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Schick T, Dwivedi-Yu J, Dessì R, Raileanu R, Lomeli M, Zettlemoyer L, Cancedda N, and Scialom T (2023). Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761. · arXiv
- Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, Scales N, Tanwani A, Cole-Lewis H, Pfohl S, et al. (2022). Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138. · arXiv
- Wei J, Tay Y, Bommasani R, Raffel C, Zoph B, Borgeaud S, Yogatama D, Bosma M, Zhou D, Metzler D, et al. (2022a). Emergent abilities of large language models. arXiv preprint arXiv:2206.07682. · arXiv
- Wei J, Wang X, Schuurmans D, Bosma M, Chi E, Le Q, and Zhou D (2022b). Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. · arXiv
- Wong C, Zheng S, Gu Y, Moung C, Abel J, Usuyama N, Weerasinghe R, Piening B, Naumann T, Bifulco C, et al. (2023). Scaling clinical trial matching using large language models: A case study in oncology. arXiv preprint arXiv:2308.02180. · arXiv
- Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan K, and Cao Y (2022). React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. · arXiv
- Yuan J, Tang R, Jiang X, and Hu X (2023). Llm for patient-trial matching: Privacy-aware data augmentation towards better performance and generalizability. arXiv preprint arXiv:2303.16756. · arXiv
Cited by
Sorted by citations
The TRIPOD-LLM reporting guideline for studies using large language models
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03425-5 · Cited by 615 · see citing works
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad493 · Cited by 359 · see citing works
Self-citationImproving large language models for clinical named entity recognition via prompt engineering
Hu, Y., Chen, Q., Du, J., Peng, X., Keloth, V. K., Zuo, X., Zhou, Y., Li, Z., Jiang, X., Lu, Z., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad259 · Cited by 314 · see citing works
Self-citationMatching patients to clinical trials with large language models
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z. (2024) · Nature Communications · doi:10.1038/s41467-024-53081-z · Cited by 287 · see citing works
Self-citationRetrieval augmented generation for large language models in healthcare: A systematic review
Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000877 · Cited by 267 · see citing works
Rood, J. E., Hupalowska, A., & Regev, A. (2024) · Cell · doi:10.1016/j.cell.2024.07.035 · Cited by 139 · see citing works
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141 · see citing works
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Tang, X., Zou, A., Zhang, Z., Li, Z., Zhao, Y., Zhang, X., Cohan, A., & Gerstein, M. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.33 · Cited by 133 · see citing works
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16789 · Cited by 71 · see citing works
“, 2023), multimodal functionalities (Gupta & Kembhavi, 2023), and specialized skills in vertical domains (Jin et al., 2023).”
BackgroundShi, W., Xu, R., Zhuang, Y., Yu, Y., Zhang, J., Wu, H., Zhu, Y., Ho, J. C., Yang, C., & Wang, M. D. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1245 · Cited by 54 · see citing works
Leveraging generative AI for clinical evidence synthesis needs to ensure trustworthiness
Zhang, G., Jin, Q., Jered McInerney, D., Chen, Y., Wang, F., Cole, C. L., Yang, Q., Wang, Y., Malin, B. A., Peleg, M., et al. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104640 · Cited by 40 · see citing works
Self-citationArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes
Hong, S., Xiao, L., Zhang, X., & Chen, J. (2024) · 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822109 · Cited by 21 · see citing works
LAB-Bench: Measuring Capabilities of Language Models for Biology Research
Laurent, J. M., Janizek, J. D., Ruzo, M., Hinks, M. M., Hammerling, M. J., Narayanan, S., Ponnapati, M., White, A. D., & Rodriques, S. G. (2024) · arXiv · doi:10.48550/arxiv.2407.10362 · Cited by 21 · see citing works
“DbQA questions were designed to cover a wide array of data sources, such that a model or agent would not be able to answer them all with the use of a single API [20].”
Background
- Authors
- Jin, Q., Yang, Y., Chen, Q., & Lu, Z.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Lu, Zhiyong
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-02-01
- Volume
- 40
- Issue
- 2
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC10904143
- OpenAlex
- W4366563389
- Cited by
- 176
- References
- 4
- Citations by year
- 2026: 50 · 2025: 67 · 2024: 44 · 2023: 15
- FWCI
- 21.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- Locations
- Bioinformatics (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- PubMed Central (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- NIH
- National Library of Medicine
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, in-context learning, hallucination mitigation, multi-hop question answering, codices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
