ProteinBERT: a universal deep-learning model of protein sequence and function
Open accessPublished version of a preprint
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Summary: Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. However, existing models and pretraining methods are designed and optimized for text analysis. We introduce ProteinBERT, a deep language model specifically designed for proteins. Our pretraining scheme combines language modeling with a novel task of Gene Ontology (GO) annotation prediction. We introduce novel architectural elements that make the model highly efficient and flexible to long sequences. The architecture of ProteinBERT consists of both local and global representations, allowing end-to-end processing of these types of inputs and outputs. ProteinBERT obtains near state-of-the-art performance, and sometimes exceeds it, on multiple benchmarks covering diverse protein properties (including protein structure, post-translational modifications and biophysical attributes), despite using a far smaller and faster model than competing deep-learning methods. Overall, ProteinBERT provides an efficient framework for rapidly training protein predictors, even with limited labeled data. Availability and implementation: Code and pretrained model weights are available at https://github.com/nadavbra/protein_bert. Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In biology applications, deep learning has been employed to learn the structure of DNA and protein sequences [6,7]”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“According to the transfer-learning paradigm, a model is first pre-trained on one task, and then fine-tuned on other downstream tasks of interest [7–9].”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Pfam: the protein families database
Finn, R. D., Bateman, A., Clements, J., Coggill, P., Eberhardt, R. Y., Eddy, S. R., Heger, A., Hetherington, K., Holm, L., Mistry, J., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1223 · Cited by 6,148 · cited 1×
“To this day, protein research is still dominated by classical sequence-similarity methods (such as BLAST [18] and hidden Markov models [19]), in contrast to domains such as computer vision which have become dominated by deep learning.”
Method
Research works printed but not linked: 2
- Abadi, 2016, 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI16), 265
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
Cited by
ProtGPT2 is a deep unsupervised language model for protein design
Ferruz, N., Schmidt, S., & Höcker, B. (2022) · Nature Communications · doi:10.1038/s41467-022-32007-7 · Cited by 795
DeepLoc 2.0: multi-label subcellular localization prediction using protein language models
Thumuluri, V., Almagro Armenteros, J. J., Johansen, A. R., Nielsen, H., & Winther, O. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac278 · Cited by 723
ProGen2: Exploring the boundaries of protein language models
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.002 · Cited by 480
Genome-wide prediction of disease variant effects with a deep protein language model
Brandes, N., Goldman, G., Wang, C. H., Ye, C. J., & Ntranos, V. (2023) · Nature Genetics · doi:10.1038/s41588-023-01465-0 · Cited by 421
Self-citationHow to build the virtual cell with artificial intelligence: Priorities and opportunities
Bunne, C., Roohani, Y., Rosen, Y., Gupta, A., Zhang, X., Roed, M., Alexandrov, T., AlQuraishi, M., Brennan, P., Burkhardt, D. B., et al. (2024) · Cell · doi:10.1016/j.cell.2024.11.015 · Cited by 393
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349
“…(ProtTrans [127]), protein sequence generation (ProGen [128]), protein function prediction (ProtST [129]), major posttranslational modification prediction (ProteinBERT [130]), biophysical property prediction (PromptProtein [131]), and advancing the state of the art in proteomics [132,133].”
BepiPred ‐3.0: Improved B‐cell epitope prediction using protein language models
Clifford, J. N., Høie, M. H., Deleuran, S., Peters, B., Nielsen, M., & Marcatili, P. (2022) · Protein Science · doi:10.1002/pro.4497 · Cited by 296
Scientific Large Language Models: A Survey on Biological & Chemical Domains
Zhang, Q., Ding, K., Lv, T., Wang, X., Yin, Q., Zhang, Y., Yu, J., Wang, Y., Li, X., Xiang, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3715318 · Cited by 122
Machine learning for functional protein design
Notin, P., Rollins, N., Gal, Y., Sander, C., & Marks, D. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 270
Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering
Yang, J., Li, F.-Z., & Arnold, F. H. (2024) · ACS Central Science · doi:10.1021/acscentsci.3c01275 · Cited by 246
Foundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 247
Chen, B., Cheng, X., Li, P., Geng, Y., Gong, J., Li, S., Bei, Z., Tan, X., Wang, B., Zeng, X., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02636-z · Cited by 240
RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks
Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., & Šikić, M. (2025) · Nature Communications · doi:10.1038/s41467-025-60872-5 · Cited by 122
Born, J., & Manica, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00639-z · Cited by 219
Rapid in silico directed evolution by a protein language model with EVOLVEpro
Jiang, K., Yan, Z., Di Bernardo, M., Sgrizzi, S. R., Villiger, L., Kayabolen, A., Kim, B. J., Carscadden, J. K., Hiraizumi, M., Nishimasu, H., et al. (2025) · Science · doi:10.1126/science.adr6006 · Cited by 205
Zhang, Y., Chen, X., Jin, B., Wang, S., Ji, S., Wang, W., & Han, J. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.498 · Cited by 44
CellSAM: a foundation model for cell segmentation
Marks, M., Israel, U., Dilip, R., Li, Q., Yu, C., Laubscher, E., Iqbal, A., Pradhan, E., Ates, A., Abt, M., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02879-w · Cited by 204
ProteInfer, deep neural networks for protein functional inference
Sanderson, T., Bileschi, M. L., Belanger, D., & Colwell, L. J. (2023) · eLife · doi:10.7554/elife.80942 · Cited by 195
Xiao, Q., Liu, K., Hong, Y., & Tang, H. (2024) · ACS In Focus · doi:10.1021/acsinfocus.7e8012 · Cited by 190
Transformer-based deep learning for predicting protein properties in the life sciences
Chandra, A., Tünnermann, L., Löfstedt, T., & Gratz, R. (2023) · eLife · doi:10.7554/elife.82819 · Cited by 184
Foundation models and intelligent decision-making: Progress, challenges, and perspectives
Huang, J., Xu, Y., Wang, Q., Wang, Q., Liang, X., Wang, F., Zhang, Z., Wei, W., Zhang, B., Huang, L., et al. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100948 · Cited by 186
Applications of transformer-based language models in bioinformatics: a survey
Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023) · Bioinformatics Advances · doi:10.1093/bioadv/vbad001 · Cited by 176
Bioinformatics approaches to discovering food-derived bioactive peptides: Reviews and perspectives
Du, Z., Comer, J. R., & Li, Y. (2023) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2023.117051 · Cited by 174
AI-Driven Deep Learning Techniques in Protein Structure Prediction
Chen, L., Li, Q., Nasif, K. F. A., Xie, Y., Deng, B., Niu, S., Pouriyeh, S., Dai, Z., Chen, J., & Xie, Y. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25158426 · Cited by 172
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2024) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 169
Transformer models in biomedicine
Madan, S., Lentzen, M., Brandt, J., Rueckert, D., Hofmann‐Apitius, M., & Fröhlich, H. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02600-5 · Cited by 172
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Nguyen, E., Poli, M., Faizi, M., Thomas, A., Birch-Sykes, C., Wornow, M., Patel, A., Rabideau, C., Massaroli, S., Bengio, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.15794 · Cited by 168
Foundation models in bioinformatics
Guo, F., Guan, R., Li, Y., Liu, Q., Wang, X., Yang, C., & Wang, J. (2025) · National Science Review · doi:10.1093/nsr/nwaf028 · Cited by 110
Wang, D., Feng, L., Ye, J., Zou, J., & Zheng, Y. (2023) · MedComm – Future Medicine · doi:10.1002/mef2.43 · Cited by 120
Convolutions are competitive with transformers for protein sequence pretraining
Yang, K. K., Fusi, N., & Lu, A. X. (2024) · Cell Systems · doi:10.1016/j.cels.2024.01.008 · Cited by 137
NetSolP: predicting protein solubility in Escherichia coli using language models
Thumuluri, V., Martiny, H.-M., Almagro Armenteros, J. J., Salomon, J., Nielsen, H., & Johansen, A. R. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btab801 · Cited by 115
Akbar, R., Bashour, H., Rawat, P., Robert, P. A., Smorodina, E., Cotet, T.-S., Flem-Karlsen, K., Frank, R., Mehta, B. B., Vu, M. H., et al. (2022) · mAbs · doi:10.1080/19420862.2021.2008790 · Cited by 113
Large language models for drug discovery and development
Zheng, Y., Koh, H. Y., Ju, J., Yang, M., May, L. T., Webb, G. I., Li, L., Pan, S., & Church, G. M. (2025) · Patterns · doi:10.1016/j.patter.2025.101346 · Cited by 133
When will RNA get its AlphaFold moment?
Schneider, B., Sweeney, B. A., Bateman, A., Cerny, J., Zok, T., & Szachniuk, M. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad726 · Cited by 114
CSM-Toxin: A Web-Server for Predicting Protein Toxicity
Morozov, V., Rodrigues, C. H. M., & Ascher, D. B. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15020431 · Cited by 124
Hyperbolic Graph Neural Networks: A Review of Methods and ApplicationsPreprint
Yang, M., Zhou, M., Zhang, T., Liu, J., Li, Z., Pan, L., Xiong, H., & King, I. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.13852 · Cited by 122
Raza, A., Uddin, J., Almuhaimeed, A., Akbar, S., Zou, Q., & Ahmad, A. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01563 · Cited by 121
ACP-MHCNN: an accurate multi-headed deep-convolutional neural network to predict anticancer peptides
Ahmed, S., Muhammod, R., Khan, Z. H., Adilina, S., Sharma, A. K., Shatabda, S., & Dehzangi, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-02703-3 · Cited by 119
Large language models and their applications in bioinformatics
Sarumi, O. A., & Heider, D. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.09.031 · Cited by 119
Akbar, S., Zou, Q., Raza, A., & Alarfaj, F. K. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102860 · Cited by 119
The Large Language Models on Biomedical Data Analysis: A Survey
Lan, W., Tang, Z., Liu, M., Chen, Q., Peng, W., Chen, Y. P., & Pan, Y. F. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3530794 · Cited by 113
Structure-informed Language Models Are Protein Designers
Zheng, Z., Deng, Y., Xue, D., Zhou, Y., Ye, F., & Gu, Q. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.03.526917 · Cited by 62
Multiple sequence alignment-based RNA language model and its application to structural inference
Zhang, Y., Lang, M., Jiang, J., Gao, Z., Xu, F., Litfin, T., Chen, K., Singh, J., Huang, X., Song, G., et al. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkad1031 · Cited by 112
Albanese, K. I., Barbe, S., Tagami, S., Woolfson, D. N., & Schiex, T. (2025) · Nature Reviews Methods Primers · doi:10.1038/s43586-025-00383-1 · Cited by 114
ProSST: Protein Language Modeling with Quantized Structure and Disentangled Attention
Hong, L., Li, M., Ma, X., Ouyang, W., Tan, P., Tan, Y., Yu, H., Zhong, B., Zhou, B., & Zhou, Z. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1126 · Cited by 44
Diffusion Language Models Are Versatile Protein LearnersPreprint
Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., & Gu, Q. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.18567 · Cited by 111
Masked inverse folding with sequence transfer for protein representation learning
(2022) · doi:10.1093/protein/gzad015 · Cited by 110
PLASMe: a tool to identify PLASMid contigs from short-read assemblies using transformer
Tang, X., Shang, J., Ji, Y., & Sun, Y. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad578 · Cited by 107
Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers
Abdine, H., Chatzianastasis, M., Bouyioukos, C., & Vazirgiannis, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.28948 · Cited by 104
ProLLaMA: A Protein Large Language Model for Multitask Protein Language Processing
Lv, L., Lin, Z.-Y., Li, H., Liu, Y.-Y., Cui, J.-X., Chen, C., Yuan, L., & Tian, Y.-H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3564914 · Cited by 109
PanGu-Coder: Program Synthesis with Function-Level Language ModelingPreprint
Christopoulou, F., Λάμπουρας, Γ., Gritta, M., Zhang, G., Guo, Y., Li, Z., Zhang, Q., Xiao, M., Shen, B., Li, L., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.11280 · Cited by 103
Prediction of Klebsiella phage-host specificity at the strain level
Boeckaerts, D., Stock, M., Ferriol-González, C., Oteo-Iglesias, J., Sanjuán, R., Domingo-Calap, P., De Baets, B., & Briers, Y. (2024) · Nature Communications · doi:10.1038/s41467-024-48675-6 · Cited by 80
Computational scoring and experimental evaluation of enzymes generated by neural networks
Johnson, S. R., Fu, X., Viknander, S., Goldin, C., Monaco, S., Zelezniak, A., & Yang, K. K. (2025) · Nature Biotechnology · doi:10.1038/s41587-024-02214-2 · Cited by 99
Generalized biological foundation model with unified nucleic acid and protein language
He, Y., Fang, P., Shan, Y., Pan, Y., Wei, Y., Chen, Y., Chen, Y., Liu, Y., Zeng, Z., Zhou, Z. M., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01044-4 · Cited by 102
Qiu, Y., & Wei, G.-W. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad289 · Cited by 88
Deep language models for interpretative and predictive materials science
Hu, Y., & Buehler, M. J. (2023) · APL Machine Learning · doi:10.1063/5.0134317 · Cited by 78
Toward real-world automated antibody design with combinatorial Bayesian optimization
Khan, A. A., Cowen-Rivers, A. I., Grosnit, A., Deik, D.-G.-X., Robert, P. A., Greiff, V., Smorodina, E., Rawat, P., Akbar, R., Dreczkowski, K., et al. (2023) · Cell Reports Methods · doi:10.1016/j.crmeth.2022.100374 · Cited by 96
Advancing microbial production through artificial intelligence-aided biology
Gong, X., Zhang, J., Gan, Q., Teng, Y., Hou, J., Lyu, Y., Liu, Z., Wu, Z., Dai, R., Zou, Y., et al. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108399 · Cited by 93
Large language models improve annotation of prokaryotic viral proteins
Flamholz, Z. N., Biller, S. J., & Kelly, L. (2024) · Nature Microbiology · doi:10.1038/s41564-023-01584-8 · Cited by 92
Machine Learning Empowering Drug Discovery: Applications, Opportunities and Challenges
Qi, X., Zhao, Y., Qi, Z., Hou, S., & Chen, J. (2024) · Molecules · doi:10.3390/molecules29040903 · Cited by 92
Progress and opportunities of foundation models in bioinformatics
Li, Q., Hu, Z., Wang, Y., Li, L., Fan, Y., King, I., Jia, G., Wang, S., Song, L., & Li, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae548 · Cited by 88
AI Methods for Antimicrobial Peptides: Progress and Challenges
Brizuela, C. A., Liu, G., Stokes, J. M., & de la Fuente‐Núñez, C. (2025) · Microbial Biotechnology · doi:10.1111/1751-7915.70072 · Cited by 87
PTM-Mamba: a PTM-aware protein language model with bidirectional gated Mamba blocks
Peng, F. Z., Wang, C., Chen, T., Schussheim, B., Vincoff, S., & Chatterjee, P. (2025) · Nature Methods · doi:10.1038/s41592-025-02656-9 · Cited by 69
Roche, R., Moussad, B., Shuvo, M. H., Tarafder, S., & Bhattacharya, D. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae039 · Cited by 85
Structure‐Based Drug Discovery with Deep Learning**
Özçelik, R., van Tilborg, D., Jiménez-Luna, J., & Grisoni, F. (2023) · ChemBioChem · doi:10.1002/cbic.202200776 · Cited by 86
Application of Transformers in Cheminformatics
Luong, K.-D., & Singh, A. K. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c02070 · Cited by 85
Deep learning for protein structure prediction and design—progress and applications
Jänes, J., & Beltrão, P. (2024) · Molecular Systems Biology · doi:10.1038/s44320-024-00016-x · Cited by 84
Bhat, S., Palepu, K., Hong, L., Mao, J., Ye, T., Iyer, R., Zhao, L., Chen, T., Vincoff, S., Watson, R., et al. (2025) · Science Advances · doi:10.1126/sciadv.adr8638 · Cited by 82
Leveraging transformers‐based language models in proteome bioinformatics
Le, N. Q. K. (2023) · PROTEOMICS · doi:10.1002/pmic.202300011 · Cited by 81
The past, present and future of de novo protein design
Yang, W., Wang, S., Lee, G. R., Zhang, J. Z., Courbet, A., Juergens, D., Wang, X., Schlichthaerle, T., Abedi, M. H., Ragotte, R. J., et al. (2026) · Nature · doi:10.1038/s41586-026-10328-7 · Cited by 82
Large language models in plant biology
Lam, H. Y. I., Ong, X. E., & Mutwil, M. (2024) · Trends in Plant Science · doi:10.1016/j.tplants.2024.04.013 · Cited by 77
Shahid, Hayat, M., Alghamdi, W., Akbar, S., Raza, A., Kadir, R. A., & Sarker, M. R. (2025) · Scientific Reports · doi:10.1038/s41598-024-84146-0 · Cited by 76
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75
Protein Structure Prediction: Challenges, Advances, and the Shift of Research Paradigms
Huang, B., Kong, L., Wang, C., Ju, F., Zhang, Q., Zhu, J., Gong, T., Zhang, H., Yu, C., Zheng, W., et al. (2023) · Genomics Proteomics & Bioinformatics · doi:10.1016/j.gpb.2022.11.014 · Cited by 74
Novel machine learning approaches revolutionize protein knowledge
Bordin, N., Dallago, C., Heinzinger, M., Kim, S., Littmann, M., Rauer, C., Steinegger, M., Rost, B., & Orengo, C. (2023) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2022.11.001 · Cited by 72
“…leveraging deep learning teaching machines to encode information contained in billions of known protein sequences by adapting so-called language models (LMs) from natural language processing (NLP) to learn aspects of the ‘ grammar ’ of the language of life as encoded in protein sequences [46 – 51].”
BackgroundDPLM-2: A Multimodal Diffusion Protein Language ModelPreprint
Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., & Gu, Q. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.13782 · Cited by 72
Su, P., & Vijay‐Shanker, K. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04642-w · Cited by 71
Advancing bioinformatics with language models: components, applications, and perspectives
Liu, J., Yang, M., Yu, Y., Xu, H., Wang, T., Li, K., & Zhou, X. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag367 · Cited by 72
Cross-disciplinary perspectives on the potential for artificial intelligence across chemistry
Mroz, A. M., Basford, A. R., Hastedt, F., Jayasekera, I. S., Mosquera‐Lois, I., Sedgwick, R., Ballester, P. J., Bocarsly, J. D., del Rio‐Chanona, E. A., Evans, M. L., et al. (2025) · Chemical Society Reviews · doi:10.1039/d5cs00146c · Cited by 72
A multimodal conversational agent for DNA, RNA and protein tasks
de Almeida, B. P., Richard, G., Dalla-Torre, H., Blum, C. F., Hexemer, L., Pandey, P., Laurent, S., Rajesh, C., Lopez, M., Laterre, A., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01047-1 · Cited by 71
A scoping review of artificial intelligence applications in clinical trial risk assessment
Teodoro, D., Naderi, N., Yazdani, A., Zhang, B., & Bornet, A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01886-7 · Cited by 70
Deep learning in structural bioinformatics: current applications and future perspectives
Kumar, N., & Srivastava, R. K. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae042 · Cited by 70
DTI-LM: language model powered drug–target interaction prediction
Ahmed, K. T., Ansari, M. I., & Zhang, W. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae533 · Cited by 69
Technological advancements in antibody-based therapeutics for treatment of diseases
Lu, R., Chiang, H.-L., Yuan, J. P., Wang, H.-H., Chen, C.-Y., Panda, S. S., Liang, K.-H., Peng, H., Ko, S.-H., Hsu, H.-J., et al. (2025) · Journal of Biomedical Science · doi:10.1186/s12929-025-01190-2 · Cited by 68
ProteinNPT: Improving Protein Property Prediction and Design with Non-Parametric Transformers
Notin, P., Weitzman, R., Marks, D. S., & Gal, Y. (2023) · neural information processing systems · doi:10.52202/075280-1458 · Cited by 24
Linguistically inspired roadmap for building biologically reliable protein language models
Vu, M. H., Akbar, R., Robert, P. A., Swiatczak, B., Sandve, G. K., Greiff, V., & Haug, D. T. T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00637-1 · Cited by 54
Akbar, S., Ullah, M., Raza, A., Zou, Q., & Alghamdi, W. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01758 · Cited by 65
Recent Advances in Generative AI for Healthcare Applications
Shokrollahi, Y., Colmenarez, J., Liu, W., Yarmohammadtoosky, S., Nikahd, M. M., Dong, P., Li, X., & Gu, L. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01908-0 · Cited by 66
BERT-like Pre-training for Symbolic Piano Music Classification TasksPreprint
Chou, Y.-H., Chen, I., Chang, C.-J., Ching, J., & Yang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05223 · Cited by 64
PoET: A generative model of protein families as sequences-of-sequences
Truong, T. F., & Bepler, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.06156 · Cited by 28
Using machine learning to predict the effects and consequences of mutations in proteins
Diaz, D. J., Kulikova, A. V., Ellington, A. D., & Wilke, C. O. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2022.102518 · Cited by 52
Enzyme function and evolution through the lens of bioinformatics
Ribeiro, A. J. M., Riziotis, I. G., Borkakoti, N., & Thornton, J. M. (2023) · Biochemical Journal · doi:10.1042/bcj20220405 · Cited by 59
Protein language models are biased by unequal sequence sampling across the tree of life
Ding, F., & Steinhardt, J. (2024) · doi:10.1101/2024.03.07.584001 · Cited by 57
Luo, Z., Wang, R., Sun, Y., Liu, J., Chen, Z., & Zhang, Y. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad534 · Cited by 57
Drug-target interaction/affinity prediction: Deep learning models and advances review
Vefghi, A., Rahmati, Z., & Akbari, M. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110438 · Cited by 58
Lin, A., Ye, J., Qi, C., Zhu, L., Mou, W., Gan, W., Zeng, D., Tang, B., Xiao, M., Chu, G., et al. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf357 · Cited by 58
Deep self-supervised learning for biosynthetic gene cluster detection and product classification
Rios-Martinez, C., Bhattacharya, N., Amini, A. P., Crawford, L. A., & Yang, K. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011162 · Cited by 56
Latent L-systems: Transformer-based Tree Generator
Lee, J. J., Li, B., & Beneš, B. (2023) · ACM Transactions on Graphics · doi:10.1145/3627101 · Cited by 55
Latent generative landscapes as maps of functional diversity in protein sequence space
Ziegler, C., Martin, J., Sinner, C., & Morcos, F. (2023) · Nature Communications · doi:10.1038/s41467-023-37958-z · Cited by 42
Akbar, S., Raza, A., Awan, H. H., Zou, Q., Alghamdi, W., & Saeed, A. (2025) · ACS Omega · doi:10.1021/acsomega.4c11449 · Cited by 55
Metadata
- Authors
- Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M.
- Affiliations
- Hebrew University of Jerusalem (IL)
- Deep Trading Ltd. , Haifa 3508401, Israel
- Ben-Gurion University of the Negev (IL)
- Department of Biological Chemistry, The Alexander Silberman Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem 9190401, Israel
- Corresponding author
- Brandes, Nadav
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-02-10
- Volume
- 38
- Issue
- 8
- Pages
- 2102-2110
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC9386727
- OpenAlex
- W4205773061
- Cited by
- 1,009
- References
- 44
- Citations by year
- 2026: 248 · 2025: 355 · 2024: 241 · 2023: 127 · 2022: 34
- FWCI
- 62.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Israel Science Foundation (2753/20)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- protein sequence representation, protein function prediction, protein structure prediction, long sequence modeling
- MeSH terms
- Deep Learning, Amino Acid Sequence, Language, Natural Language Processing, Proteins
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.