ProteinBERT: a universal deep-learning model of protein sequence and function
Open accessPublished version of a preprint
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Summary: Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. However, existing models and pretraining methods are designed and optimized for text analysis. We introduce ProteinBERT, a deep language model specifically designed for proteins. Our pretraining scheme combines language modeling with a novel task of Gene Ontology (GO) annotation prediction. We introduce novel architectural elements that make the model highly efficient and flexible to long sequences. The architecture of ProteinBERT consists of both local and global representations, allowing end-to-end processing of these types of inputs and outputs. ProteinBERT obtains near state-of-the-art performance, and sometimes exceeds it, on multiple benchmarks covering diverse protein properties (including protein structure, post-translational modifications and biophysical attributes), despite using a far smaller and faster model than competing deep-learning methods. Overall, ProteinBERT provides an efficient framework for rapidly training protein predictors, even with limited labeled data. Availability and implementation: Code and pretrained model weights are available at https://github.com/nadavbra/protein_bert. Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In biology applications, deep learning has been employed to learn the structure of DNA and protein sequences [6,7]”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“According to the transfer-learning paradigm, a model is first pre-trained on one task, and then fine-tuned on other downstream tasks of interest [7–9].”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Pfam: the protein families database
Finn, R. D., Bateman, A., Clements, J., Coggill, P., Eberhardt, R. Y., Eddy, S. R., Heger, A., Hetherington, K., Holm, L., Mistry, J., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1223 · Cited by 6,148 · cited 1×
“To this day, protein research is still dominated by classical sequence-similarity methods (such as BLAST [18] and hidden Markov models [19]), in contrast to domains such as computer vision which have become dominated by deep learning.”
Method
Research works printed but not linked: 2
- Abadi, 2016, 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI16), 265
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
Cited by
Items 201–300 of 1,005
DeepZF: improved DNA-binding prediction of C2H2-zinc-finger proteins by deep transfer learning
Aizenshtein-Gazit, S., & Orenstein, Y. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac469 · Cited by 26
Insights into the inner workings of transformer models for protein function prediction
Wenzel, M. A., Grüner, E., & Strodthoff, N. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae031 · Cited by 26
Protein functional site annotation using local structure embeddings
Derry, A., Tartici, A., & Altman, R. B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2513219122 · Cited by 26
A Survey of Pre-trained Language Models for Processing Scientific TextPreprint
Ho, X., Nguyen, A.-K. D., Dao, A., Jiang, J., Chida, Y., Sugimoto, K., To, H. Q., Boudin, F., & Aizawa, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.17824 · Cited by 26
Prediction of antibiotic resistance mechanisms using a protein language model
Yagimoto, K., Hosoda, S., Sato, M., & Hamada, M. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae550 · Cited by 25
Composition Based Oxidation State Prediction of Materials Using Deep Learning Language Models
Fu, N., Hu, J., Feng, Y., Morrison, G. W., Loye, H. Z., & Hu, J. (2023) · Advanced Science · doi:10.1002/advs.202301011 · Cited by 25
Liu, H., Guan, F., Liu, T., Yang, L., Fan, L., Liu, X., Luo, H., Wu, N., Yao, B., Tian, J., et al. (2023) · 中国科学通报:英文版 · doi:10.1016/j.scib.2023.09.039 · Cited by 25
Protein engineering in the deep learning era
Zhou, B., Tan, Y., Hu, Y., Zheng, L., Zhong, B., & Hong, L. (2024) · mLife · doi:10.1002/mlf2.12157 · Cited by 25
Artificial Intelligence Transforming Post-Translational Modification Research
Kim, D. N., Yin, T., Zhang, T., Im, A. K., Cort, J., Rozum, J. C., Pollock, D. D., Qian, W., & Feng, S. (2024) · Bioengineering · doi:10.3390/bioengineering12010026 · Cited by 25
A Survey on Protein Representation Learning: Retrospect and ProspectPreprint
Wu, L., Huang, Y., Lin, H., & Li, S. Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.00813 · Cited by 25
AI for Biomedicine in the Era of Large Language ModelsPreprint
Bi, Z., Dip, S. A., Hajialigol, D., Kommu, S., Liu, H., Meng, L., & Wang, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.15673 · Cited by 25
HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language ModelPreprint
Ma, M., Liu, G., Cao, C., Pan, D., Dao, T., Gu, A., Jin, P., Yang, Z., Xia, Y., Luo, R., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.10807 · Cited by 26
Genome-wide prediction of disease variants with a deep protein language model
Brandes, N., Goldman, G., Wang, C. H., Ye, C. J., & Ntranos, V. (2022) · doi:10.1101/2022.08.25.505311 · Cited by 24
“A different deep-learning approach to functionalizing variants uses protein language models, a technique originally developed for natural language processing, which has been successfully applied to many protein prediction tasks [18–23].”
MethodSelf-citationFoundation models of protein sequences: A brief overview.
Bjerregaard, A., Groth, P. M., Hauberg, S., Krogh, A., & Boomsma, W. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103004 · Cited by 18
A Survey of Large Language Model for Drug Research and Development
Guo, H., Xing, X., Zhou, Y., Jiang, W., Chen, X., Wang, T., Jiang, Z., Wang, Y., Hou, J., Jiang, Y., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3552256 · Cited by 24
An update on knowledge graphs and their current and potential applications in drug discovery
Serra, A., Fratello, M., Federico, A., & Greco, D. (2025) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2025.2490253 · Cited by 24
How generative Artificial Intelligence can transform drug discovery?
Jusoh, A. S., Remli, M. A., Mohamad, M. S., Cazenave, T., & Fong, C. (2025) · European Journal of Medicinal Chemistry · doi:10.1016/j.ejmech.2025.117825 · Cited by 24
Foundation models in plant molecular biology: advances, challenges, and future directions
Xu, F., Wu, T., Qian, C., Wang, X., & Yan, J. (2025) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1611992 · Cited by 24
Nguyen, T.-T., Jiang, Z., Nguyen, V., Le, N. Q. K., & Chua, M. C. H. (2025) · ACS Omega · doi:10.1021/acsomega.5c05484 · Cited by 24
Protein Large Language Models: A Comprehensive Survey
Xiao, Y., Zhao, W., Zhang, J., Jin, Y., Zhang, H., Ren, Z., Sun, R., Wang, H., Wan, G., Pan, L., et al. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1255 · Cited by 24
Language model and its interpretability in biomedicine: A scoping review
Lyu, D., Wang, X., Chen, Y., & Wang, F. (2024) · iScience · doi:10.1016/j.isci.2024.109334 · Cited by 26
Cross-domain contrastive graph neural network for lncRNA–protein interaction prediction
Li, H., Wu, B., Sun, M., Zhu, Z., Chen, K., & Ge, H. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111901 · Cited by 23
Awwaliyah, N. K., & Fatimah, M. (2024) · Jurnal Pendidikan dan Pembelajaran Indonesia (JPPI) · doi:10.53299/jppi.v4i3.621 · Cited by 23
PTM-Mamba: A PTM-Aware Protein Language Model with Bidirectional Gated Mamba Blocks
Peng, Z. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3680276 · Cited by 23
Yu, C., Fang, X., Tian, S., & Liu, H. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00973-w · Cited by 23
Sun, Z., Xu, J., Zhang, Y., Zhang, Y., Wang, Z., Wang, X., Li, S., Guo, Y., Shen, H., & Song, J. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf261 · Cited by 23
Elucidating the functional roles of prokaryotic proteins using big data and artificial intelligence
Ardern, Z., Chakraborty, S., Lenk, F., & Kaster, A. (2023) · FEMS Microbiology Reviews · doi:10.1093/femsre/fuad003 · Cited by 23
Malik, M. S., & Ou, Y. (2023) · Methods · doi:10.1016/j.ymeth.2023.10.008 · Cited by 23
When Protein Structure Embedding Meets Large Language Models
Ali, S., Chourasia, P., & Patterson, M. (2023) · Genes · doi:10.3390/genes15010025 · Cited by 23
Prediction of Protein–Protein Interactions Based on Integrating Deep Learning and Feature Fusion
Tran, H.-N., Nguyen, P.-X.-Q., Guo, F., & Wang, J. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25115820 · Cited by 23
EGPDI: identifying protein–DNA binding sites based on multi-view graph embedding fusion
Zheng, M., Sun, G., Li, X., & Fan, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae330 · Cited by 23
Frank, M., Ni, P., Jensen, M., & Gerstein, M. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2320510121 · Cited by 23
Li, M., Shi, Y., Hu, S., Hu, S., Guo, P., Wan, W., Zhang, L. Y., Pan, S., Li, J., Sun, L., et al. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae579 · Cited by 23
Yan, C., Zhang, Z., Xu, J., Yu, M., Yan, S., Wei, L., Zou, Q., Zhang, Q., & Cui, F. (2025) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2025.142309 · Cited by 23
From text to traits: exploring the role of large language models in plant breeding
Yoosefzadeh-Najafabadi, M. (2025) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1583344 · Cited by 23
Highly accurate assembly polishing with DeepPolisher
Mastoras, M., Asri, M., Brambrink, L., Hebbar, P., Kolesnikov, A., Cook, D. E., Nattestad, M., Lucas, J., Won, S. T., Chang, P.-C., et al. (2025) · Genome Research · doi:10.1101/gr.280149.124 · Cited by 22
From machine learning to multimodal models: The AI revolution in enzyme engineering
Shi, Z., Xu, S., Xue, S., Chen, K., Lu, Y., Wang, F., Long, S., Tian, Y., Zhang, P., Wang, J., et al. (2025) · BioDesign Research · doi:10.1016/j.bidere.2025.100044 · Cited by 22
Recent Approaches for Bioactive Peptides Production from Pulses and Pseudocereals
Martoccia, M., Disca, V., Jaouhari, Y., Coïsson, J. D., & Bordiga, M. (2025) · Molecules · doi:10.3390/molecules30214304 · Cited by 22
Graff, D., Pyzer‐Knapp, E. O., Jordan, K. E., Shakhnovich, E. I., & Coley, C. W. (2023) · Digital Discovery · doi:10.1039/d3dd00088e · Cited by 22
Ngo, N. K., & Hy, T. (2024) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/ad3ee4 · Cited by 18
Variant Effect Prediction in the Age of Machine Learning
Bromberg, Y., Prabakaran, R., Kabir, A., & Shehu, A. (2024) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041467 · Cited by 23
Review and Comparative Analysis of Methods and Advancements in Predicting Protein Complex Structure
Zhao, N., Wu, T., Wang, W., Zhang, L., & Gong, X. (2024) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-024-00626-x · Cited by 22
E1: Retrieval-Augmented Protein Encoder ModelsPreprint
Jain, S., Beazer, J., Ruffolo, J. A., Bhatnagar, A., & Madani, A. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.12.688125 · Cited by 22
Chen, C., Xu, Y., Ouyang, J., Xiong, X., Łabaj, P. P., Chmielarczyk, A., Różańska, A., Zhang, H., Liu, K., Shi, T., et al. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf271 · Cited by 22
Nayar, G., Tartici, A., & Altman, R. B. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013424 · Cited by 21
Exploring the Protein Sequence Space with Global Generative Models
Romero‐Romero, S., Lindner, S., & Ferruz, N. (2023) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041471 · Cited by 21
Wang, X. R., Zhou, W., Wang, R., & Gao, X. (2024) · Symmetry · doi:10.3390/sym16040464 · Cited by 21
Deep learning in template-free de novo biosynthetic pathway design of natural products
Xie, X., Gui, L., Qiao, B., Wang, G., Huang, S., Zhao, Y., & Sun, S. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae495 · Cited by 21
Asim, M. N., Ibrahim, M. A., Asif, T., & Dengel, A. R. (2025) · Heliyon · doi:10.1016/j.heliyon.2024.e41488 · Cited by 21
Integrating genetic algorithms and language models for enhanced enzyme design
Teukam, Y. G. N., Zipoli, F., Laino, T., Criscuolo, E., Grisoni, F., & Manica, M. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae675 · Cited by 21
TLCrys: Transfer Learning Based Method for Protein Crystallization Prediction
Jin, C., Shi, Z., Kang, C., Lin, K. Y., & Zhang, H. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23020972 · Cited by 21
Advances of Deep Learning in Protein Science: A Comprehensive SurveyPreprint
Hu, B.-Z., Tan, C., Wu, L.-R., Zheng, J.-B., Xia, J., Gao, Z.-Y., Liu, Z.-C., Wu, F., Zhang, G., & Li, S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.05314 · Cited by 14
Semantical and geometrical protein encoding toward enhanced bioactivity and thermostability
Tan, Y., Zhou, B., Zheng, L., Fan, G., & Hong, L. (2025) · eLife · doi:10.7554/elife.98033.4 · Cited by 20
Local large language model‐assisted literature mining for on‐surface reactions
Xiang, J., Li, Y., Zhang, X., He, Y., & Sun, Q. (2025) · Materials Genome Engineering Advances · doi:10.1002/mgea.88 · Cited by 20
Enzyme functional classification using artificial intelligence
Kim, H. R., Ji, H., Kim, G. B., & Lee, S. Y. (2025) · Trends in biotechnology · doi:10.1016/j.tibtech.2025.03.003 · Cited by 21
ProtNote: a multimodal method for protein–function annotation
Char, S., Corley, N., Alamdari, S., Yang, K., & Amini, A. P. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf170 · Cited by 20
Artificial intelligence for nanomedicine
Song, X., Gao, X., Wang, H., Yu, F., Qin, M., Li, Y., Liu, Y., Feng, W., Zhou, C., Chukavin, N. N., et al. (2025) · Science China Chemistry · doi:10.1007/s11426-025-2942-5 · Cited by 21
Ten simple rules for optimal and careful use of generative AI in science
Helmy, M., Jin, L., Alhossary, A., Mansour, T. A., Pellegrina, D. V. D. S., & Selvarajoo, K. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013588 · Cited by 20
Functional annotation of proteins for signaling network inference in non-model species
den Broeck, L. V., Bhosale, D. K., Song, K., de Lima, C. F. F., Ashley, M. C. B., Zhu, T., Zhu, S., van de Cotte, B., Neyt, P., Ortiz, A. C., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-40365-z · Cited by 20
Bitter peptide prediction using graph neural networks
Srivastava, P., Steuer, A., Ferri, F., Nicoli, A., Schultz, K., Bej, S., Pizio, A. D., & Wolkenhauer, O. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00909-x · Cited by 20
FAPM: functional annotation of proteins using multimodal models beyond structural modeling
Xiang, W., Xiong, Z., Chen, H., Xiong, J., Zhang, W., Fu, Z., Zheng, M., Liu, B., & Shi, Q. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae680 · Cited by 20
Alsaedi, S., Gao, X., & Gojobori, T. (2025) · FEBS Open Bio · doi:10.1002/2211-5463.70003 · Cited by 17
Mardikoraem, M., & Woldring, D. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15051337 · Cited by 20
ESM ‐scan—A tool to guide amino acid substitutions
Totaro, M. G., Vide, U., Zausinger, R., Winkler, A., & Oberdorfer, G. (2024) · Protein Science · doi:10.1002/pro.5221 · Cited by 19
Exploring a diverse world of effector domains and amyloid signaling motifs in fungal NLR proteins
Wojciechowski, J. W., Tekoglu, E., Gąsior-Głogowska, M., Coustou, V., Szulc, N., Szefczyk, M., Kopaczyńska, M., Saupe, S. J., & Dyrka, W. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010787 · Cited by 19
Language models for protein design
Lee, J. S., Abdin, O., & Kim, P. M. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103027 · Cited by 19
Phytopathogen Effector Biology in the Burgeoning AI Era
Jones, D., & Raffaele, S. (2025) · Annual Review of Phytopathology · doi:10.1146/annurev-phyto-121823-081033 · Cited by 19
Wang, L., Fu, X., Yang, J., Zhang, X., Ye, X., Sakurai, T., Zeng, X., & Liu, Y. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf376 · Cited by 19
codonGPT: reinforcement learning on a generative language model enables scalable mRNA design
Rajbanshi, B., & Guruacharya, A. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf1345 · Cited by 19
A flaw in using pretrained protein language models in protein–protein interaction inference models
Szymborski, J., & Emad, A. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-025-01176-7 · Cited by 19
A comprehensive overview of recent advances in generative models for antibodies
Meng, F., Zhou, N., Guangchun, H., Liu, R., Zhang, Y., Jing, M., & Hou, Q. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.06.016 · Cited by 19
Advancing plant biology through deep learning-powered natural language processing
Peng, S., & Rajjou, L. (2024) · Plant Cell Reports · doi:10.1007/s00299-024-03294-9 · Cited by 19
Linguistics-based formalization of the antibody language as a basis for antibody language models
Vu, M. H., Robert, P. A., Akbar, R., Świątczak, B., Sandve, G. K., Haug, D. T. T., & Greiff, V. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00642-3 · Cited by 19
Wang, Y., Wang, Z., Sadeh, G., Zancato, L., Achille, A., Karypis, G., & Rangwala, H. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.29.620988 · Cited by 19
Distilling Structural Representations into Protein Sequence ModelsPreprint
Ouyang-Zhang, J., Gong, C., Zhao, Y., Krähenbühl, P., Klivans, A. R., & Diaz, D. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.08.622579 · Cited by 19
Liu, N., Sun, C., Ji, T., Tian, J., Tang, J., Wu, Y., & Lan, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.11618 · Cited by 19
Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment
Fei, X., Chatzianastasis, M., Carneiro, S. A., Abdine, H., Petalidis, L. P., & Vazirgiannis, M. (2025) · neural information processing systems · doi:10.52202/085713-0883 · Cited by 18
Protein–protein interaction and site prediction using transfer learning
Liu, T., Gao, H., Ren, X., Xu, G., Liu, B., Wu, N., Luo, H., Wang, Y., Tu, T., Yao, B., et al. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad376 · Cited by 18
Palacios, A. V., Acharya, P., Peidl, A. S., Beck, M. R., Blanco, E., Mishra, A., Bawa‐Khalfe, T., & Pakhrin, S. C. (2024) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqae011 · Cited by 18
Machine learning for predicting protein properties: A comprehensive review
Wang, Y., Zhang, Y., Zhan, X., He, Y., Yang, Y., Cheng, L., & Alghazzawi, D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128103 · Cited by 18
Zhang, Y., Zheng, L., You, N., Hu, W., Jiang, W., Lu, M., Xu, H., Dai, H., Fu, T., & Zhou, Y. (2025) · Journal of Pharmaceutical Analysis · doi:10.1016/j.jpha.2025.101255 · Cited by 18
PBertKla: a protein large language model for predicting human lysine lactylation sites
Lai, H., Luo, D., Yang, M., Zhu, T., Yang, H., Luo, X., Wei, Y., Xie, S., Hong, F., Shu, K., et al. (2025) · BMC Biology · doi:10.1186/s12915-025-02202-1 · Cited by 18
Reprogramming pretrained language models for protein sequence representation learning
Vinod, R., Chen, P., & Das, P. (2025) · Digital Discovery · doi:10.1039/d4dd00195h · Cited by 18
xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data
Gong, J., Hao, M., Zeng, X., Liu, C., Ma, J., Cheng, X., Wang, T., Zhang, X., & Song, L. (2023) · doi:10.1101/2023.03.24.534055 · Cited by 17
Wang, S., Bergman, M. T., Hall, C. K., & You, F. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01401 · Cited by 17
Technologies for Monoclonal Antibody Discovery and Development
Han, K. H., Li, Y., Parveen, R., Venkataraman, S., & Lin, C. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms262110470 · Cited by 17
A survey on large language models in biology and chemistry
Ashyrmamatov, I., Gwak, S. J., Jin, S.-Y., Jun, I., Ucak, U. V., Lee, J. Y., & Lee, J. (2025) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01583-1 · Cited by 17
PepBERT: Lightweight language models for bioactive peptide representation
Du, Z., Caragea, D., Guo, X., & Li, Y. (2026) · Future Foods · doi:10.1016/j.fufo.2026.100999 · Cited by 17
Knowledge Enhanced Representation Learning for Drug Discovery
Hoang, T. L., Sbodio, M. L., Galindo, M. M., Zayats, M., Fernández-Díaz, R., Valls, V., Picco, G., Berrospi, C., & López, V. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28924 · Cited by 17
Miniaturizing and modifying natural proteins with Raygun
Devkota, K., Shonai, D., Mao, Y., Ko, Y. S., Wang, W., Soderling, S. H., & Singh, R. (2026) · Nature · doi:10.1038/s41586-026-10842-8 · Cited by 17
Deep protein representations enable recombinant protein expression prediction
Martiny, H.-M., Armenteros, J. J. A., Johansen, A. R., Salomon, J., & Nielsen, H. (2021) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2021.107596 · Cited by 17
Lam, H. T., Sbodio, M. L., Gallindo, M. M., Zayats, M., Fernández-Díaz, R., Valls, V., Picco, G., Berrospi, C., & López, V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.12802 · Cited by 17
Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent SpacePreprint
Lee, M.-J., Vecchietti, L. F., Jung, H., Ro, H.-J., Cha, M., & Kim, H. M. (2024) · International Conference on Machine Learning · doi:10.48550/arxiv.2405.18986 · Cited by 22
Advances in Protein Representation Learning: Methods, Applications, and Future DirectionsPreprint
Nguyen, V. T. D., & Hy, T.-S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.16659 · Cited by 17
Elucidating the Design Space of Multimodal Protein Language ModelsPreprint
Hsieh, C.-Y., Wang, X., Zhang, D., Xue, D., Ye, F., Huang, S., Zheng, Z., & Gu, Q. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.11454 · Cited by 17
Balancing speed and precision in protein folding: a comparison of AlphaFold2, ESMFold, and OmegaFold
Hýsková, A., Maršálková, E., & Šimeček, P. (2026) · Frontiers in Genetics · doi:10.3389/fgene.2025.1715037 · Cited by 16
Hu, J., Chen, K., Rao, B., Ni, J.-Y., Thafar, M. A., Albaradei, S., & Arif, M. (2024) · Analytical Biochemistry · doi:10.1016/j.ab.2024.115637 · Cited by 16
Ashikhmina, M. S., Zenkin, A. M., Pantiukhin, I. S., Litvak, I. G., Nesterov, P. V., Dutta, K., Shityakov, S., Nosonovsky, M., Korablev-Dyson, M. A., Orlova, O. Y., et al. (2024) · Food Bioscience · doi:10.1016/j.fbio.2024.105358 · Cited by 16
Effective Gene Expression Prediction and Optimization from Protein Sequences
Liu, T., Zhang, Y., Li, Y., Xu, G., Gao, H., Wang, P., Tu, T., Luo, H., Wu, N., Yao, B., et al. (2025) · Advanced Science · doi:10.1002/advs.202407664 · Cited by 16
Enhancing nucleotide sequence representations in genomic analysis with contrastive optimization
Refahi, M. S., Sokhansanj, B. A., Mell, J. C., Brown, J. R., Yoo, H., Hearne, G., & ROSEN, G. L. (2025) · Communications Biology · doi:10.1038/s42003-025-07902-6 · Cited by 16
Metadata
- Authors
- Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M.
- Affiliations
- Hebrew University of Jerusalem (IL)
- Deep Trading Ltd. , Haifa 3508401, Israel
- Ben-Gurion University of the Negev (IL)
- Department of Biological Chemistry, The Alexander Silberman Institute of Life Sciences, The Hebrew University of Jerusalem, Jerusalem 9190401, Israel
- Corresponding author
- Brandes, Nadav
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-02-10
- Volume
- 38
- Issue
- 8
- Pages
- 2102-2110
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC9386727
- OpenAlex
- W4205773061
- Cited by
- 1,009
- References
- 44
- Citations by year
- 2026: 248 · 2025: 355 · 2024: 241 · 2023: 127 · 2022: 34
- FWCI
- 62.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Israel Science Foundation (2753/20)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- protein sequence representation, protein function prediction, protein structure prediction, long sequence modeling
- MeSH terms
- Deep Learning, Amino Acid Sequence, Language, Natural Language Processing, Proteins
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.