The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
MSA Transformer
Open accessAbstract
Abstract Unsupervised protein language models trained across millions of diverse sequences learn structure and function of proteins. Protein language models studied to date have been trained to perform inference from individual sequences. The longstanding approach in computational biology has been to make inferences from a family of evo lutionarily related sequences by fitting a model to each family independently. In this work we combine the two paradigms. We introduce a protein language model which takes as input a set of sequences in the form of a multiple sequence alignment. The model interleaves row and column attention across the input sequences and is trained with a variant of the masked language modeling objective across many protein families. The performance of the model surpasses current state-of-the-art unsupervised structure learning methods by a wide margin, with far greater parameter efficiency than prior state-of-the-art protein language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We used the same set of clusters to make predictions using the unsupervised deep learning model MSA Transformer36 and found that these clusters contained evolutionary couplings for both states, and the score based on contact maps correlated with the root mean squared deviation (r.m.s.d.) in AF2 (Methods and Extended Data Fig. 3).”
From Predicting multiple conformations via sequence clustering and AlphaFold2 (2024) · cites this paper as [36]
In the citing paper
“MSA transformer is an unsupervised learning method, which signifies that its contact predictions purely reflect evolutionary couplings learned in sequences, rather than being supervised on structure as is the case AF2.”
From Predicting multiple conformations via sequence clustering and AlphaFold2 (2024) · cites this paper as [36]
In the citing paper
“Among many successful neural network architectures, attention-based networks are extremely powerful for modeling complicated interactions between entities, making them suitable for modeling proteins whose functions are largely determined by interactions between amino acids (34–37).”
From Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization (2022) · cites this paper as [36]
In the citing paper
“In the MSA Transformer (16), the neural network model exploits the equivalence of aligned positions in the MSA in order to derive embeddings via a tied row attention framework.”
From Ultrafast end-to-end protein structure prediction enables high-throughput exploration of uncharacterised proteins (2020) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280
Self-citationRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434
Improved protein structure prediction using predicted interresidue orientations
Yang, J., Anishchenko, I., Park, H., Peng, Z., Ovchinnikov, S., & Baker, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1914677117 · Cited by 1,421
HH-suite3 for fast remote homology detection and deep protein annotation
Steinegger, M., Meier, M., Mirdita, M., Vöhringer, H., Haunsberger, S. J., & Söding, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3019-7 · Cited by 1,145
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131
Research works printed but not linked: 5
- 10.1002/(SICI)1097-0134(19990301)34:4508::AID-PROT103.0.CO;2-4 · DOI
- Ho, J. , Kalchbrenner, N. , Weissenborn, D. , and Salimans, T. Axial Attention in Multidimensional Transformers. arXiv, 12 2019. URL http://arxiv.org/abs/1912.12180. · arXiv
- Sturmfels, P. , Vig, J. , Madani, A. , and Rajani, N. F. Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models. bioRxiv, 11 2020. URL http://arxiv.org/abs/2012.00195. · arXiv
- Thomas, J. , Ramakrishnan, N. , and Bailey-Kellogg, C. Graphical models of residue coupling in protein families, 4 2008. ISSN 15455963. URL https://pubmed.ncbi.nlm.nih.gov/18451428/.
- 2017, Attention is All you Need, Advances in Neural Information Processing Systems, 30, 5998
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Self-citationLanguage models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.07.09.450648 · Cited by 635
Self-citationPredicting multiple conformations via sequence clustering and AlphaFold2
Wayment-Steele, H. K., Ojoawo, A., Otten, R., Apitz, J. M., Pitsawong, W., Hömberger, M., Ovchinnikov, S., Colwell, L., & Kern, D. (2024) · Nature · doi:10.1038/s41586-023-06832-9 · Cited by 545
Scaffolding protein functional sites using deep learning
Wang, J., Lisanza, S., Juergens, D., Tischer, D., Watson, J. L., Castro, K. M., Ragotte, R., Saragovi, A., Milles, L. F., Baek, M., et al. (2022) · Science · doi:10.1126/science.abn2100 · Cited by 487
Single-sequence protein structure prediction using language models from deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., & AlQuraishi, M. (2021) · doi:10.1101/2021.08.02.454840 · Cited by 455
Single-sequence protein structure prediction using a language model and deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01432-w · Cited by 455
Learning inverse folding from millions of predicted structuresPreprint
Hsu, C., Verkuil, R., Liu, J. J., Lin, Z., Hie, B., Sercu, T., Lerer, A. K., & Rives, A. W. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.10.487779 · Cited by 492
Self-citationProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 424
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 361
Machine learning for functional protein design
Notin, P., Rollins, N. J., Gal, Y., Sander, C., & Marks, D. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 304
Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering
Yang, J., Li, F.-Z., & Arnold, F. H. (2024) · ACS Central Science · doi:10.1021/acscentsci.3c01275 · Cited by 268
Elnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 348
Machine learning in protein structure prediction
AlQuraishi, M. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.04.005 · Cited by 263
SaProt: Protein Language Modeling with Structure-aware VocabularyPreprint
Su, J., Han, C., Zhou, Y., Shan, J., Zhou, X., & Yuan, F. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.01.560349 · Cited by 264
Antibody structure prediction using interpretable deep learning
Ruffolo, J. A., Sulam, J., & Gray, J. J. (2022) · Patterns · doi:10.1016/j.patter.2021.100406 · Cited by 216
Informed training set design enables efficient machine learning-assisted directed protein evolution
Wittmann, B. J., Yue, Y., & Arnold, F. H. (2021) · Cell Systems · doi:10.1016/j.cels.2021.07.008 · Cited by 215
(2023) · doi:10.1101/2023.12.07.570727 · Cited by 207
Prihoda, D., Maamary, J., Waight, A., Juan, V., Fayadat-Dilman, L., Svozil, D., & Bitton, D. A. (2022) · mAbs · doi:10.1080/19420862.2021.2020203 · Cited by 190
The landscape of tolerated genetic variation in humans and primates
(2023) · doi:10.1126/science.abn8197 · Cited by 182
Goshisht, M. K. (2024) · ACS Omega · doi:10.1021/acsomega.3c05913 · Cited by 173
Shan, S., Luo, S., Yang, Z., Hong, J., Su, Y., Ding, F., Fu, L., Li, C., Chen, P., Ma, J., et al. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2122954119 · Cited by 157
DNA language models are powerful predictors of genome-wide variant effects
Benegas, G., Batra, S. S., & Song, Y. S. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311219120 · Cited by 157
DeepConsensus improves the accuracy of sequences with a gap-aware sequence transformer
Baid, G., Cook, D. E., Shafin, K., Yun, T., Llinares-López, F., Berthet, Q., Belyaeva, A., Töpfer, A., Wenger, A. M., Rowell, W. J., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01435-7 · Cited by 152
Protein language model embeddings for fast, accurate, alignment-free protein structure prediction
Weissenow, K., Heinzinger, M., & Rost, B. (2021) · doi:10.1101/2021.07.31.454572 · Cited by 139
Toward the solution of the protein structure prediction problem
Pearce, R., & Zhang, Y. (2021) · Journal of Biological Chemistry · doi:10.1016/j.jbc.2021.100870 · Cited by 132
Transformer models in biomedicine
Madan, S., Lentzen, M., Brandt, J., Rueckert, D., Hofmann‐Apitius, M., & Fröhlich, H. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02600-5 · Cited by 146
Fang, X., Wang, F., Liu, L., He, J., Lin, D., Xiang, Y., Zhu, K., Zhang, X., Wu, H., Li, H., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00721-6 · Cited by 132
A DNA language model based on multispecies alignment predicts the effects of genome-wide variants
Benegas, G., Albors, C., Aw, A. J., Ye, C., & Song, Y. S. (2025) · Nature Biotechnology · doi:10.1038/s41587-024-02511-w · Cited by 128
Deep generative modeling for protein design
Strokach, A. V., & Kim, P. M. (2021) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2021.11.008 · Cited by 122
NetSolP: predicting protein solubility in Escherichia coli using language models
Thumuluri, V., Martiny, H.-M., Armenteros, J. J. A., Salomon, J., Nielsen, H., & Johansen, A. R. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab801 · Cited by 118
Deep-learning-based single-domain and multidomain protein structure prediction with D-I-TASSER
Zheng, W. H., Wuyun, Q., Li, Y., Liu, Q., Zhou, X., Peng, C., Zhu, Y., Freddolino, L., & Zhang, Y. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02654-4 · Cited by 129
Predicting functional effect of missense variants using graph attention neural networks
Zhang, H., Xu, M. S., Fan, X., Chung, W. K., & Shen, Y. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00561-w · Cited by 98
AlQuraishi, M., & Sorger, P. K. (2021) · Nature Methods · doi:10.1038/s41592-021-01283-4 · Cited by 96
Sato, K., & Hamada, M. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad186 · Cited by 104
In silico proof of principle of machine learning-based antibody design at unconstrained scale
Akbar, R., Robert, P. A., Weber, C. R., Widrich, M., Frank, R., Pavlović, M., Scheffer, L., Chernigovskaya, M., Snapkov, I., Slabodkin, A., et al. (2022) · mAbs · doi:10.1080/19420862.2022.2031482 · Cited by 94
Tao, J., Chen, Z., Sun, Z., Guo, H., Leng, B., Yu, Z., Wang, Y., He, Z., Lei, X., & Yang, J. (2023) · Remote Sensing · doi:10.3390/rs15061602 · Cited by 85
Ding, K., Chin, M., Zhao, Y., Huang, W., Mai, B. K., Wang, H., Liu, P., Yang, Y., & Luo, Y. (2024) · Nature Communications · doi:10.1038/s41467-024-50698-y · Cited by 86
Enhancing efficiency of protein language models with minimal wet-lab data through few-shot learning
Zhou, Z., Zhang, L., Yu, Y., Wu, B., Li, M., Hong, L., & Tan, P. (2024) · Nature Communications · doi:10.1038/s41467-024-49798-6 · Cited by 93
Representation learning applications in biological sequence analysis
Iuchi, H., Matsutani, T., Yamada, K., Iwano, N., Sumi, S., Hosoda, S., Zhao, S., Fukunaga, T., & Hamada, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.05.039 · Cited by 87
Boadu, F., Cao, H., & Cheng, J. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad208 · Cited by 81
Deep graph learning of inter-protein contacts
Xie, Z., & Xu, J. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btab761 · Cited by 67
Application of Transformers in Cheminformatics
Luong, K.-D., & Singh, A. K. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c02070 · Cited by 76
Self-play reinforcement learning guides protein engineering
Wang, Y., Tang, H., Huang, L., Pan, L., Yang, L., Yang, H., Mu, F., & Yang, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00691-9 · Cited by 74
A general temperature-guided language model to design proteins of enhanced stability and activity
Jiang, F., Li, M., Dong, J., Yu, Y., Sun, X., Wu, B., Huang, J., Kang, L., Pei, Y., Zhang, L., et al. (2024) · Science Advances · doi:10.1126/sciadv.adr2641 · Cited by 71
Self-citationDiaz, D. J., Gong, C., Ouyang-Zhang, J., Loy, J. M., Wells, J., Yang, D. K., Ellington, A. D., Dimakis, A. G., & Klivans, A. R. (2024) · Nature Communications · doi:10.1038/s41467-024-49780-2 · Cited by 69
Buess, L., Keicher, M., Navab, N., Maier, A., & Arasteh, S. T. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00497-1 · Cited by 79
Notin, P., van Niekerk, L., Kollasch, A. W., Ritter, D., Gal, Y., & Marks, D. S. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.07.519495 · Cited by 60
Fast and effective protein model refinement using deep graph neural networks
Jing, X., & Xu, J. (2021) · Nature Computational Science · doi:10.1038/s43588-021-00098-9 · Cited by 52
A text-guided protein design framework
Liu, S., Li, Y., Li, Z., Gitter, A., Zhu, Y., Lu, J., Xu, Z., Nie, W., Ramanathan, A., Xiao, C., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01011-z · Cited by 97
Livesey, B. J., & Marsh, J. A. (2025) · Genome Biology · doi:10.1186/s13059-025-03575-w · Cited by 50
Lu, A. X., Lu, A. X., Pritišanac, I., Zarin, T., Forman-Kay, J. D., & Moses, A. M. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010238 · Cited by 57
Zeng, W., Dou, Y., Pan, L., Xu, L., & Peng, S. (2024) · Nature Communications · doi:10.1038/s41467-024-52293-7 · Cited by 49
Effect of tokenization on transformers for biological sequences
Dotan, E., Jaschek, G., Pupko, T., & Belinkov, Y. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae196 · Cited by 49
What does AlphaFold3 learn about antibody and nanobody docking, and what remains unsolved?
Hitawala, F. N., & Gray, J. J. (2025) · mAbs · doi:10.1080/19420862.2025.2545601 · Cited by 88
Xia, Y., Zhao, K., Liu, D., Zhou, X., & Zhang, G. (2023) · Communications Biology · doi:10.1038/s42003-023-05610-7 · Cited by 48
An end-to-end framework for the prediction of protein structure and fitness from single sequence
Chen, Y., Xu, Y., Liu, D., Xing, Y., & Gong, H. (2024) · Nature Communications · doi:10.1038/s41467-024-51776-x · Cited by 45
Protein sequence‐to‐structure learning: Is this the end(‐to‐end revolution)?
Laine, E., Eismann, S., Elofsson, A., & Grudinin, S. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26235 · Cited by 44
“Recent works have also shown that the residue-residue dependencies extracted by certain attention heads in transformers trained on large amounts of sequences can be directly interpreted as 3D contacts or distances [133, 134, 62].”
BackgroundMethodHe, H., Chen, G., Tang, Z., & Chen, C. Y. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01464-x · Cited by 43
ADOPT: intrinsic protein disorder prediction through deep bidirectional transformers
Redl, I., Fisicaro, C., Dutton, O., Hoffmann, F., Henderson, L. D., Owens, B. M. J., Heberling, M. M., Paci, E., & Tamiola, K. (2023) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad041 · Cited by 42
What does AlphaFold3 learn about antigen and nanobody docking, and what remains unsolved?Preprint
Hitawala, F. N., & Gray, J. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.09.21.614257 · Cited by 41
Lin, P., Yan, Y., & Huang, S. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac499 · Cited by 40
Zero‐shot mutation effect prediction on protein stability and function using RoseTTAFold
Mansoor, S., Baek, M., Juergens, D., Watson, J. L., & Baker, D. A. (2023) · Protein Science · doi:10.1002/pro.4780 · Cited by 45
Understanding the brain with attention: A survey of transformers in brain sciences
Chen, C., Wang, H., Chen, Y., Yin, Z., Yang, X., Ning, H., Zhang, Q., Li, W., Xiao, R., & Zhao, J. (2023) · Brain‐X · doi:10.1002/brx2.29 · Cited by 44
Protein Engineering with Lightweight Graph Denoising Neural Networks
Zhong, B., Zheng, L., Wu, B., Tan, Y., Lv, O., Yi, K., Fan, G., & Hong, L. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c00036 · Cited by 39
Xu, Y., Liu, D., & Gong, H. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00716-2 · Cited by 41
Engineering highly active nuclease enzymes with machine learning and high-throughput screening
Thomas, N., Belanger, D., Xu, C. A., Lee, H., Hirano, K., Iwai, K., Polic, V., Nyberg, K. D., Hoff, K. G., Frenz, L., et al. (2025) · Cell Systems · doi:10.1016/j.cels.2025.101236 · Cited by 41
CpG Transformer for imputation of single-cell methylomes
De Waele, G., Clauwaert, J., Menschaert, G., & Waegeman, W. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btab746 · Cited by 37
Chandra, A. A., Sharma, A. K., Dehzangi, A., Tsunoda, T., & Sattar, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-47624-5 · Cited by 38
Rapid directed evolution guided by protein language models and epistatic interactions
Tran, V. Q., Nemeth, M., Bartie, L. J., Chandrasekaran, S. S., Fanton, A., Moon, H. C., Hie, B., Konermann, S., & Hsu, P. D. (2026) · Science · doi:10.1126/science.aea1820 · Cited by 41
Meta learning addresses noisy and under-labeled data in machine learning-guided antibody engineering
Minot, M., & Reddy, S. T. (2024) · Cell Systems · doi:10.1016/j.cels.2023.12.003 · Cited by 35
Zhang, Y., Zhang, Y., Xiong, Y., Wang, H., Deng, Z., Song, J., & Ou, H. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab420 · Cited by 31
PerturbNet predicts single-cell responses to unseen chemical and genetic perturbations
Yu, H., Qian, W., Song, Y., & Welch, J. D. (2025) · Molecular Systems Biology · doi:10.1038/s44320-025-00131-3 · Cited by 57
Accurate and robust protein sequence design with CarbonDesign
Ren, M., Yu, C., Bu, D., & Zhang, H. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00838-2 · Cited by 30
Chen, X., Morehead, A., Liu, J., & Cheng, J. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad203 · Cited by 29
Self-citationSeq-InSite: sequence supersedes structure for protein interaction site prediction
Hosseini, S., Golding, G. B., & Ilie, L. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btad738 · Cited by 26
Engineering a New Generation of Gene Editors: Integrating Synthetic Biology and AI Innovations
Chia, B. S., Seah, S., Wang, B., Shen, K., Srivastava, D., & Chew, W. L. (2025) · ACS Synthetic Biology · doi:10.1021/acssynbio.4c00686 · Cited by 31
Lab-in-the-loop therapeutic antibody design with deep learningPreprint
Frey, N. C., Hötzel, I., Stanton, S., Kelly, R. L., Alberstein, R. G., Makowski, E. K., Martinkus, K., Berenberg, D., Bevers, J. J., Bryson, T., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.19.639050 · Cited by 33
tFold-Ab: Fast and Accurate Antibody Structure Prediction without Sequence HomologsPreprint
Wu, J., Wu, F., Jiang, B., Liu, W., & Zhao, P. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.11.10.515918 · Cited by 26
Identification of bacteriophage genome sequences with representation learning
Bai, Z., Zhang, Y., Miyano, S., Yamaguchi, R., Fujimoto, K., Uematsu, S., & Imoto, S. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac509 · Cited by 27
Si, Y., & Yan, C. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad039 · Cited by 25
Improved prediction of MHC-peptide binding using protein language models
Hashemi, N., Hao, B., Ignatov, M., Paschalidis, I. C., Vakili, P., Vajda, S., & Kozakov, D. (2023) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2023.1207380 · Cited by 25
CLADE 2.0: Evolution-Driven Cluster Learning-Assisted Directed Evolution
Qiu, Y., & Wei, G. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01046 · Cited by 23
SSEmb: A joint embedding of protein sequence and structure enables robust variant effect predictions
Blaabjerg, L. M., Jonsson, N., Boomsma, W., Stein, A., & Lindorff‐Larsen, K. (2024) · Nature Communications · doi:10.1038/s41467-024-53982-z · Cited by 31
Zhang, C., Wang, Q., Li, Y., Teng, A., Hu, G., Wuyun, Q., & Zheng, W. H. (2024) · Biomolecules · doi:10.3390/biom14121531 · Cited by 29
Li, Y., Zhang, C., Zheng, W. H., Zhou, X., Bell, E. W., Yu, D., & Zhang, Y. (2021) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26211 · Cited by 24
pLM4CPPs: Protein Language Model-Based Predictor for Cell Penetrating Peptides
Kumar, N., Du, Z., & Li, Y. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01338 · Cited by 22
PRECOGx: e x ploring GPCR signaling mechanisms with deep protein representations
Matic, M., Singh, G., Carli, F., Rosa, N. D. O., Miglionico, P., Magni, L., Gutkind, J. S., Russell, R. B., Inoue, A., & Raimondi, F. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac426 · Cited by 22
An integrative approach to protein sequence design through multiobjective optimization
Hong, L., & Kortemme, T. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011953 · Cited by 25
Machine Learning for Protein Engineering
Johnston, K. E., Fannjiang, C., Wittmann, B. J., Hie, B., Yang, K., & Wu, Z. (2023) · Challenges and advances in computational chemistry and physics · doi:10.1007/978-3-031-37196-7_9 · Cited by 25
Abbasi, K., Razzaghi, P., Gharizadeh, A., Ghareyazi, A., Dehnad, A., Rabiee, H. R., & Mofrad, M. R. K. (2025) · Pharmacological Reviews · doi:10.1016/j.pharmr.2025.100095 · Cited by 24
Boosting single-cell gene regulatory network reconstruction via bulk-cell transcriptomic data
Shu, H., Ding, F., Zhou, J., Xue, Y., Zhao, D. Y., Zeng, J., & Ma, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac389 · Cited by 20
PITHIA: Protein Interaction Site Prediction Using Multiple Sequence Alignments and Attention
Hosseini, S., & Ilie, L. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms232112814 · Cited by 20
Tan, Y., Li, M., Zhou, Z., Tan, P., Yu, H., Fan, G., & Liang, H. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00884-3 · Cited by 22
Non-Identifiability and the Blessings of Misspecification in Models of Molecular Fitness
Weinstein, E. N., Amin, A., Frazer, J., & Marks, D. S. (2022) · neural information processing systems · doi:10.52202/068431-0396 · Cited by 19
Phyloformer: towards fast and accurate phylogeny estimation with self-attention networksPreprint
Nesterenko, L., Boussau, B., & Jacob, L. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.06.24.496975 · Cited by 20
A-Prot: protein structure modeling using MSA transformer
Hong, Y., Lee, J., & Ko, J. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04628-8 · Cited by 19
Metadata
- Authors
- Rao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A.
- Affiliations
- Facility for Antiproton and Ion Research (DE)
- Meta (United States) (US)
- University of California, Berkeley (US)
- Corresponding author
- Rao, Roshan
- Rives, Alexander
- ORCID
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2021-02-13
- OpenAlex
- W3133458480
- Cited by
- 350
- References
- 47
- Citations by year
- 2026: 63 · 2025: 90 · 2024: 74 · 2023: 45 · 2022: 47 · 2021: 28
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- protein language models, multiple sequence alignment, masked language modeling, protein structure prediction, parameter efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
