Foundation models in bioinformatics
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
With the adoption of foundation models (FMs), artificial intelligence (AI) has become increasingly significant in bioinformatics and has successfully addressed many historical challenges, such as pre-training frameworks, model evaluation and interpretability. FMs demonstrate notable proficiency in managing large-scale, unlabeled datasets, because experimental procedures are costly and labor intensive. In various downstream tasks, FMs have consistently achieved noteworthy results, demonstrating high levels of accuracy in representing biological entities. A new era in computational biology has been ushered in by the application of FMs, focusing on both general and specific biological issues. In this review, we introduce recent advancements in bioinformatics FMs employed in a variety of downstream tasks, including genomics, transcriptomics, proteomics, drug discovery and single-cell analysis. Our aim is to assist scientists in selecting appropriate FMs in bioinformatics, according to four model types: language FMs, vision FMs, graph FMs and multimodal FMs. In addition to understanding molecular landscapes, AI technology can establish the theoretical and practical foundation for continued innovation in molecular biology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024) · Nature · doi:10.1038/s41586-024-07487-w · Cited by 15,861
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Gene Expression Omnibus: NCBI gene expression and hybridization array data repository
Edgar, R. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.1.207 · Cited by 13,288
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · cited 1×
“…AlexNet [6 ] is a convolutional neural network (CNN) that has significantly advanced computer vision (CV); ResNet [7 ] introduces shortcut connections with residual layers trained on ImageNet; the segment anything model [8 ] is a promptable segmentation method that segments everything every w here.”
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092 · cited 1×
“In genomics, The Cancer Genome Atlas (TCGA) [91 ] analyzes more than 20 0 0 0 cancer samples matched to normal samples, covering 33 different cancer types; TargetFinder [92 ] provides a pipeline for identifying or characterizing gene targets of distal enhancers; ArrayExpress [93 ] contains data…”
Research works printed but not linked: 62
- Mikolov, 2013, Proceedings of the 27th International Conference on Neural Information Processing Systems, 3111
- Vaswani, 2017, Proceedings of the 31st International Conference on Neural Information Processing Systems, 6000
- Devlin, 2019, Proceedings of NAACL-HLT 2019, 4171
- Gilmer, 2017, Proceedings of the 34th International Conference on Machine Learning, 1263
- Ying, 2021, Do transformers really perform badly for graph representation?, 28877
- Chen, 2024, Foundation models for bioinformatics, Quant Biol, 12, 339, 10.1002/qub2.69 · DOI
- Sarumi, 2024, Large language models and their applications in bioinformatics, Computat Struct Biotech, 23, 3498, 10.1016/j.csbj.2024.09.031 · DOI
- Guo, 2024, Diffusion models in bioinformatics and computational biology, Nat Rev Bioeng, 2, 136, 10.1038/s44222-023-00114-9 · DOI · Paper page
- Li, 2024, Progress and opportunities of foundation models in bioinformatics, Brief Bioinform, 25, bbae548, 10.1093/bib/bbae548 · DOI
- Ji, 2021, DNABERT: pre-trained bidirectional encoder representations from transformers model for DNA-language in genome, Bioinformatics, 37, 2112, 10.1093/bioinformatics/btab083 · DOI · Paper page
- Nguyen, 2023, Proceedings of the 37th International Conference on Neural Information Processing Systems, 43177
- Li, 2024, VQDNA: unleashing the power of vector quantization for multi-species genomic sequence modeling, 28717
- Schiff, 2024, Caduceus: bi-directional equivariant long-range DNA sequence modeling, 43632
- Zhou, 2015, Predicting effects of noncoding variants with deep learning–based sequence model, Nat Methods, 12, 931, 10.1038/nmeth.3547 · DOI · Paper page
- Luo, 2022, iEnhancer-BERT: a novel transfer learning architecture based on DNA-language model for identifying enhancers and their strength, 153, 10.1007/978-3-031-13829-4_13 · DOI
- Zhang, 2023, Deep flanking sequence engineering for efficient promoter design using DeepSEED, Nat Commun, 14, 6309, 10.1038/s41467-023-41899-y · DOI · Paper page
- Yu, 2021, iDNA-ABT: advanced deep learning model for detecting DNA methylation with adaptive features and transductive information maximization, Bioinformatics, 37, 4603, 10.1093/bioinformatics/btab677 · DOI
- Jin, 2022, iDNA-ABF: multi-scale deep biological language learning model for the interpretable prediction of DNA methylations, Genome Biol, 23, 219, 10.1186/s13059-022-02780-1 · DOI · Paper page
- Ni, 2023, DNA 5-methylcytosine detection and methylation phasing using pacbio circular consensus sequencing, Nat Commun, 14, 4054, 10.1038/s41467-023-39784-9 · DOI · Paper page
- Zhang, 2024, Multiple sequence alignment-based RNA language model and its application to structural inference, Nucleic Acids Res, 52, e3, 10.1093/nar/gkad1031 · DOI · Paper page
- Sumi, 2024, Deep generative design of rna family sequences, Nat Methods, 21, 435, 10.1038/s41592-023-02148-8 · DOI
- Zhao, 2024, GenerRNA: a generative pre-trained language model for de novo RNA design, PLoS One, 19, e0310814, 10.1371/journal.pone.0310814 · DOI
- Akiyama, 2022, Informative RNA base embedding for RNA structural alignment and clustering by deep representation learning, NAR Genom Bioinform, 4, lqac012, 10.1093/nargab/lqac012 · DOI · Paper page
- Chen, Self-supervised learning on millions of pre-mRNA sequences improves sequence-based RNA splicing prediction, 10.1101/2023.01.31.526427 · DOI
- Zhang, 2021, BERT-m7G: a transformer architecture based on bert and stacking ensemble to identify RNA N7-methylguanosine sites from sequence information, Comput Math Methods Med, 2021, 7764764
- Soylu, 2023, BERT2OME: prediction of 2-O-methylation modifications from RNA sequence by transformer architecture based on bert, IEEE/ACM Trans Comput Biol Bioinform, 20, 2177, 10.1109/TCBB.2023.3237769 · DOI
- Rao, 2021, MSA transformer, 8844
- Krapp, 2023, PeSTo: parameter-free geometric deep learning for accurate prediction of protein binding interfaces, Nat Commun, 14, 2175, 10.1038/s41467-023-37701-8 · DOI · Paper page
- Jiang, 2025, Rapid in silico directed evolution by a protein language model with EVOLVEpro, Science, 387, eadr6006, 10.1126/science.adr6006 · DOI · Paper page
- Wang, 2019, SMILES-BERT: large scale unsupervised pre-training for molecular property prediction, 429, 10.1145/3307339.3342186 · DOI · Paper page
- Xue, 2022, X-MOL: Large-scale pre-training for molecular understanding and diverse molecular analysis, Sci Bull, 67, 899, 10.1016/j.scib.2022.01.029 · DOI
- Xiong, 2021, ADMETlab 2.0: an integrated online platform for accurate and comprehensive predictions of ADMET properties, Nucleic Acids Res, 49, W5, 10.1093/nar/gkab255 · DOI · Paper page
- Wu, 2022, Knowledge-based BERT: a method to extract molecular features like computational chemists, Brief Bioinform, 23, bbac131, 10.1093/bib/bbac131 · DOI
- Xia, Mole-BERT: rethinking pre-training graph neural networks for molecules, 10.26434/chemrxiv-2023-dngg4 · DOI
- Li, 2023, A knowledge-guided pre-training framework for improving molecular representation learning, Nat Commun, 14, 7568, 10.1038/s41467-023-43214-1 · DOI · Paper page
- Wang, 2022, Molecular contrastive learning of representations via graph neural networks, Nat Mach Intell, 4, 279, 10.1038/s42256-022-00447-x · DOI · Paper page
- Liu, 2023, Multi-modal molecule structure–text model for text-based retrieval and editing, Nat Mach Intell, 5, 1447, 10.1038/s42256-023-00759-6 · DOI · Paper page
- Bagal, 2021, Molgpt: Molecular generation using a transformer-decoder model, J Chem Inf Model, 62, 2064, 10.1021/acs.jcim.1c00600 · DOI · Paper page
- Peng, 2022, Pocket2Mol: efficient molecular sampling based on 3D protein pockets, 17644
- Huang, 2024, A dual diffusion model enables 3D molecule generation and lead optimization based on target pockets, Nat Commun, 15, 2657, 10.1038/s41467-024-46569-1 · DOI
- Munson, 2024, De novo generation of multi-target compounds using deep generative chemistry, Nat Commun, 15, 3636, 10.1038/s41467-024-47120-y · DOI
- Bai, 2023, Interpretable bilinear attention network with domain adaptation improves drug–target prediction, Nat Mach Intell, 5, 126, 10.1038/s42256-022-00605-1 · DOI · Paper page
- Yang, 2024, Interaction-based inductive bias in graph neural networks: enhancing protein–ligand binding affinity predictions from 3D structures, IEEE Trans Pattern Anal Mach Intell, 46, 8191, 10.1109/TPAMI.2024.3400515 · DOI
- Liu, 2023, Cross-species cell-type assignment from single-cell RNA-seq data by a heterogeneous graph neural network, Genome Res, 33, 96, 10.1101/gr.276868.122 · DOI
- Li, 2022, A deep generative model for multi-view profiling of single-cell RNA-seq and ATAC-seq data, Genome Biol, 23, 10.1186/s13059-021-02595-6 · DOI
- Cao, 2024, scButterfly: a versatile single-cell cross-modality translation method via dual-aligned variational autoencoders, Nat Commun, 15, 2973, 10.1038/s41467-024-47418-x · DOI
- Schaar, Nicheformer: a foundation model for single-cell and spatial omics, 10.1101/2024.04.15.589472 · DOI
- Choi, CELLama: foundation model for single cell and spatial transcriptomics by cell embedding leveraging language model abilities, 10.1101/2024.05.08.593094 · DOI
- Wang, 2024, MarsGT: multi-omics analysis for rare population inference using single-cell graph transformer, Nat Commun, 15, 338, 10.1038/s41467-023-44570-8 · DOI
- Li, 2024, scPROTEIN: a versatile deep graph contrastive learning framework for single-cell proteomics embedding, Nat Methods, 21, 623, 10.1038/s41592-024-02214-9 · DOI
- Ma, 2023, Single-cell biological network inference using a heterogeneous graph transformer, Nat Commun, 14, 964, 10.1038/s41467-023-36559-0 · DOI · Paper page
- Drost, 2024, Multi-modal generative modeling for joint analysis of single-cell t cell receptor and gene expression data, Nat Commun, 15, 5577, 10.1038/s41467-024-49806-9 · DOI
- Tang, 2023, SiGra: single-cell spatial elucidation through an image-augmented graph transformer, Nat Commun, 14, 5618, 10.1038/s41467-023-41437-w · DOI · Paper page
- Sarkans, 2018, The biostudies database–one stop shop for all data supporting a life sciences study, Nucleic Acids Res, 46, D1266, 10.1093/nar/gkx965 · DOI · Paper page
- de Souza, 2012, The encode project, Nat Methods, 9, 1046, 10.1038/nmeth.2238 · DOI
- Burley, 2023, RCSB Protein Data Bank (RCSB.org): delivery of experimentally-determined PDB structures alongside one million computed structure models of proteins from artificial intelligence/machine learning, Nucleic Acids Res, 51, D488, 10.1093/nar/gkac1077 · DOI · Paper page
- Zdrazil, 2024, The ChEMBL database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods, Nucleic Acids Res, 52, D1180, 10.1093/nar/gkad1004 · DOI · Paper page
- Moreno, 2022, Expression atlas update: gene and protein expression in multiple species, Nucleic Acids Res, 50, D129, 10.1093/nar/gkab1030 · DOI · Paper page
- Fang, 2023, Knowledge graph-enhanced molecular contrastive learning with functional prompt, Nat Mach Intell, 5, 542, 10.1038/s42256-023-00654-0 · DOI · Paper page
- He, UltraEval: a lightweight platform for flexible and comprehensive evaluation for LLMs, 10.18653/v1/2024.acl-demos.23 · DOI
- Feng, 2024, Unveiling potential threats: backdoor attacks in single-cell pre-trained models, Cell Discov, 10, 122, 10.1038/s41421-024-00753-1 · DOI
- Wu, 2023, A causality inspired framework for model interpretation, 2731, 10.1145/3580305.3599240 · DOI
Cited by
Single-cell foundation models: bringing artificial intelligence into cell biology
Baek, S., Song, K., & Lee, I. (2025) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01547-5 · Cited by 46
“These architectures enable the development of foundation models de fi ned as large-scale, self-supervised arti fi cial intelligence (AI) models trained on diverse datasets that can be adapted for a wide range of downstream tasks 1 – 3 .”
Nallapareddy, M. V., Craighero, F., Worpenberg, L., Naef, F., Gobet, C., & Vandergheynst, P. (2025) · Communications Biology · doi:10.1038/s42003-025-09092-7 · Cited by 1
Kulkarni Kale, U., & Vemulapalli, G. (2025) · The Cancer Journal · doi:10.1097/ppo.0000000000000799 · Cited by 1
RNALens: Study on 5’ UTR Modeling and Cell-Specificity
Mao, L., Tian, Y., Qian, K.-W., & Song, Y. (2025) · doi:10.1101/2025.07.20.665722 · Cited by 1
Metadata
- Authors
- Guo, F., Guan, R., Li, Y., Liu, Q., Wang, X., Yang, C., & Wang, J.
- Venue
- National Science Review
- Publisher
- Oxford University Press (OUP)
- Published
- 2025-01-25
- Volume
- 12
- Issue
- 4
- Pages
- nwaf028-nwaf028
- ISSN
- 2053-714X (ISSN-L)
- 2095-5138
- OpenAlex
- W4406817136
- Cited by
- 110
- References
- 44
- FWCI
- 28.12
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- National Natural Science Foundation of China (62350004, 62332020, 62322215)
- National Key Research and Development Program of China (2021YFF1201200)
- Innovation and Technology Commission (ITCPD/17-9)
- Hong Kong University of Science and Technology (Z0428)
- Central South University
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Foundation models in bioinformatics. https://app.cochat.ai/papers/10.1093/nsr/nwaf028
- The paper
- Guo, F., Guan, R., Li, Y., Liu, Q., Wang, X., Yang, C., & Wang, J. (2025). Foundation models in bioinformatics. National Science Review. https://doi.org/10.1093/nsr/nwaf028
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.