The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
MSA Transformer
Open accessAbstract
Abstract Unsupervised protein language models trained across millions of diverse sequences learn structure and function of proteins. Protein language models studied to date have been trained to perform inference from individual sequences. The longstanding approach in computational biology has been to make inferences from a family of evo lutionarily related sequences by fitting a model to each family independently. In this work we combine the two paradigms. We introduce a protein language model which takes as input a set of sequences in the form of a multiple sequence alignment. The model interleaves row and column attention across the input sequences and is trained with a variant of the masked language modeling objective across many protein families. The performance of the model surpasses current state-of-the-art unsupervised structure learning methods by a wide margin, with far greater parameter efficiency than prior state-of-the-art protein language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We used the same set of clusters to make predictions using the unsupervised deep learning model MSA Transformer36 and found that these clusters contained evolutionary couplings for both states, and the score based on contact maps correlated with the root mean squared deviation (r.m.s.d.) in AF2 (Methods and Extended Data Fig. 3).”
From Predicting multiple conformations via sequence clustering and AlphaFold2 (2024) · cites this paper as [36]
In the citing paper
“MSA transformer is an unsupervised learning method, which signifies that its contact predictions purely reflect evolutionary couplings learned in sequences, rather than being supervised on structure as is the case AF2.”
From Predicting multiple conformations via sequence clustering and AlphaFold2 (2024) · cites this paper as [36]
In the citing paper
“Among many successful neural network architectures, attention-based networks are extremely powerful for modeling complicated interactions between entities, making them suitable for modeling proteins whose functions are largely determined by interactions between amino acids (34–37).”
From Deep learning guided optimization of human antibody against SARS-CoV-2 variants with broad neutralization (2022) · cites this paper as [36]
In the citing paper
“In the MSA Transformer (16), the neural network model exploits the equivalence of aligned positions in the MSA in order to derive embeddings via a tied row attention framework.”
From Ultrafast end-to-end protein structure prediction enables high-throughput exploration of uncharacterised proteins (2020) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280
Self-citationRetrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434
Improved protein structure prediction using predicted interresidue orientations
Yang, J., Anishchenko, I., Park, H., Peng, Z., Ovchinnikov, S., & Baker, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1914677117 · Cited by 1,421
HH-suite3 for fast remote homology detection and deep protein annotation
Steinegger, M., Meier, M., Mirdita, M., Vöhringer, H., Haunsberger, S. J., & Söding, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3019-7 · Cited by 1,145
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131
Research works printed but not linked: 5
- 10.1002/(SICI)1097-0134(19990301)34:4508::AID-PROT103.0.CO;2-4 · DOI
- Ho, J. , Kalchbrenner, N. , Weissenborn, D. , and Salimans, T. Axial Attention in Multidimensional Transformers. arXiv, 12 2019. URL http://arxiv.org/abs/1912.12180. · arXiv
- Sturmfels, P. , Vig, J. , Madani, A. , and Rajani, N. F. Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models. bioRxiv, 11 2020. URL http://arxiv.org/abs/2012.00195. · arXiv
- Thomas, J. , Ramakrishnan, N. , and Bailey-Kellogg, C. Graphical models of residue coupling in protein families, 4 2008. ISSN 15455963. URL https://pubmed.ncbi.nlm.nih.gov/18451428/.
- 2017, Attention is All you Need, Advances in Neural Information Processing Systems, 30, 5998
Cited by
Sorted by citations
Items 201–300 of 355
BEACON: Benchmark for Comprehensive RNA Tasks and Language ModelsPreprint
Ren, Y., Chen, Z., Qiao, L., Jing, H., Cai, Y., Xu, S. Y., Ye, P., Ma, X., Sun, S., Yan, H.-L., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.22.600190 · Cited by 4
Efficient inference, training, and fine-tuning of protein language models
Çelik, M. H., & Xie, X. (2025) · iScience · doi:10.1016/j.isci.2025.113495 · Cited by 7
Tomilova, Y. E., Russkikh, N., Yi, I. M., Shaburova, E. V., Tomilov, V. N., Pyrinova, G. B., Brezhneva, S. O., Tikhonyuk, O. S., Gololobova, N. S., Popichenko, D. V., et al. (2024) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2024.1495267 · Cited by 3
Directed Evolution of Proteins via Bayesian Optimization in Embedding Space
Soldát, M., & Kléma, J. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822356 · Cited by 3
A Deep Learning-Based Fully Automated Vertebra Segmentation and Labeling Workflow
Lu, H., Liu, M., Yu, K., Fang, Y., Zhao, J., & Shi, Y. (2025) · British Journal of Hospital Medicine · doi:10.12968/hmed.2025.0443 · Cited by 3
A comprehensive systematic literature review of multiple sequence alignment algorithms
Sabonsolin, J. A., & Lao, D. (2026) · Discover Computing · doi:10.1007/s10791-026-09911-3 · Cited by 4
The Study of Evolution and Application Related to the Chat-GPT
Li, Y. (2023) · Highlights in Science Engineering and Technology · doi:10.54097/hset.v57i.9999 · Cited by 3
CPE-Pro: A Structure-Sensitive Deep Learning Method for Protein Representation and Origin Evaluation
Gou, W., Ge, W., Tan, Y., Li, M., Fan, G., & Yu, H. (2025) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-025-00732-4 · Cited by 3
MetaLLM: Residue-Wise Metal Ion Prediction Using Deep Transformer Model
Shishir, F. S., Sarker, B., Rahman, F., & Shomaji, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-34960-7_4 · Cited by 3
Ion channel classification through machine learning and protein language model embeddings
Ghazikhani, H., & Butler, G. (2024) · Berichte aus der medizinischen Informatik und Bioinformatik/Journal of integrative bioinformatics · doi:10.1515/jib-2023-0047 · Cited by 3
Jian, Y., Gao, C., Zeng, C., Zhao, Y., & Vosoughi, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78192-6_27 · Cited by 3
Leveraging pretrained deep protein language model to predict peptide collision cross section
Nakai-Kasai, A., Ogata, K., Ishihama, Y., & Tanaka, T. (2024) · bioRxiv · doi:10.1038/s42004-025-01540-z · Cited by 4
Vinod, R., Amini, A. P., Crawford, L. A., & Yang, K. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013925 · Cited by 2
Learning to discover medicines
Nguyen, M.-T., Nguyen, T., & Tran, T. (2022) · International Journal of Data Science and Analytics · doi:10.1007/s41060-022-00371-8 · Cited by 4
An ultra-high-throughput method for measuring biomolecular activitiesPreprint
Tu, B., Sundar, V., & Esvelt, K. M. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.03.09.483646 · Cited by 5
Zhang, Y., Bai, Z., & Imoto, S. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.05.518770 · Cited by 3
He, Y., Zhou, X., Chang, C. H., Chen, G., Liu, W., Li, G., Fan, X., Ma, Y., Yuan, F., & Chang, X. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.29.564583 · Cited by 3
Complex-based Ligand-Binding Proteins Redesign by Equivariant Diffusion-based Generative ModelsPreprint
Nguyen, V. T. D., Nguyen, N. D., & Hy, T. (2024) · bioRxiv · doi:10.1101/2024.04.17.589997 · Cited by 2
Annotation Vocabulary (Might Be) All You NeedPreprint
Hallee, L., Rafailidis, N., Horger, C., Hong, D. K., & Gleghorn, J. P. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.07.30.605924 · Cited by 7
Pang, A., Luo, Y., Zhou, J., Cai, X., Huang, L., Zhang, B., Liu, Z., & Zheng, Y. (2024) · Biotechnology Journal · doi:10.1002/biot.202400203 · Cited by 3
Environmental adaptations in metagenomes revealed by deep learning
Winder, J. C., Poulton, S., Wu, T., Möck, T., & van Oosterhout, C. (2025) · BMC Biology · doi:10.1186/s12915-025-02361-1 · Cited by 3
BEST: Basic Embedding Search Tool Enhancing Discovery of Novel Enzyme
Wu, Y., Yi, X., Tan, Y., Yu, H., Fan, G., & Zheng, G. (2025) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-025-00753-z · Cited by 2
Alphappimi: a comprehensive deep learning framework for predicting PPI-modulator interactions
Liu, D., Song, T., Wang, S., Li, X., Han, P., Wang, J., & Wang, S. (2025) · Journal of Cheminformatics · doi:10.1186/s13321-025-01077-2 · Cited by 2
Khatri, K., Levy, R. M., & Haldane, A. (2025) · Physical Review Research · doi:10.1103/g5cx-1vhj · Cited by 3
Hoffmann, U. A., Knave, A., Schuppe, A. Z., Capó‐Bauçà, S., Aguiló‐Nicolau, P., Schriever, K., Karlsson, A., Russo, M., Sporre, E., Matsuda, M., et al. (2026) · Science Advances · doi:10.1126/sciadv.aee9222 · Cited by 2
Barethiya, S., Huang, J., Stumpf, C., Liu, X., Guan, H., & Chen, J. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013728 · Cited by 4
Improving Protein Secondary Structure Prediction by Deep Language Models and Transformer Networks
Wu, T., Cheng, W., & Cheng, J. J. (2024) · Methods in molecular biology · doi:10.1007/978-1-0716-4196-5_3 · Cited by 2
Lee, J.-C., Bang, D., & Kim, S. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01255 · Cited by 2
Wang, Y., & Wang, C. (2025) · Molecules · doi:10.3390/molecules30081704 · Cited by 2
Liu, Y., Bao, T., Yin, P., Wang, S., & Wang, Y. (2025) · PLoS ONE · doi:10.1371/journal.pone.0329174 · Cited by 2
Language Modelling Techniques for Analysing the Impact of Human Genetic Variation
Hegde, M., Nebel, J., & Rahman, F. (2025) · Bioinformatics and Biology Insights · doi:10.1177/11779322251358314 · Cited by 6
Large-scale multi-omic biosequence transformers for modeling protein–nucleic acid interactions
Chen, S. F., Steele, R. J., Hocky, G. M., Lemeneh, B., Lad, S. P., & Oermann, E. K. (2026) · PLoS ONE · doi:10.1371/journal.pone.0341501 · Cited by 3
Karshenas, A., Röschinger, T., & García, H. G. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014092 · Cited by 2
Sampling protein language models for functional protein design
Darmawan, J. T., Gal, Y., & Notin, P. (2026) · Cell Systems · doi:10.1016/j.cels.2026.101714 · Cited by 4
GNN-Dom: An Unsupervised Method for Protein Domain Partition via Protein Contact Map
Wang, L., & Wang, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-23198-8_26 · Cited by 2
Rahbar, M., Chauhan, R. K., Shah, P. N., Cao, R., Si, D., & Hou, J. (2022) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3535508.3545558 · Cited by 2
PSSP-MFFNet: A Multifeature Fusion Network for Protein Secondary Structure Prediction
Chen, Y., Chen, G., & Chen, C. Y. (2024) · ACS Omega · doi:10.1021/acsomega.3c10230 · Cited by 2
Secondary Structure-Guided Novel Protein Sequence Generation with Latent Graph Diffusion
Hu, Y., Tan, Y., Han, A., Zheng, L., Hong, L., & Zhong, B. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822117 · Cited by 4
Singh, J., Singh, J., Paliwal, K. K., Busch, A. W., & Zhou, Y. (2021) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb49929.2021.9562849 · Cited by 2
Construction of a neural network energy function for protein physicsPreprint
Yang, H., Xiong, Z., & Zonta, F. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.04.26.441401 · Cited by 2
Semantical and geometrical protein encoding toward enhanced bioactivity and thermostability
Tan, Y., Zhou, B., Zheng, L., Fan, G., & Hong, L. (2025) · eLife · doi:10.7554/elife.98033 · Cited by 2
Duan, H., Li, Z., Wu, Y., Chen, W., & Xia, L. C. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.07.07.602424 · Cited by 2
Zhang, Y., Zhang, H., Li, G., Wang, H., Zhang, X., Hong, X., Zhang, T.-T., Wen, L., Zhao, Y., Jiang, J., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.09.669469 · Cited by 3
Learning Dynamic Protein Representations at Scale with DistogramsPreprint
Portal, N., Karroucha, W., Mallet, V., & Bonomi, M. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.29.702509 · Cited by 3
Sweeney, M. D., & Kang, H. M. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.04.703796 · Cited by 3
Zhou, B., Liu, S. M., Chen, L., & Dai, Q. (2024) · Current Bioinformatics · doi:10.2174/0115748936343410241008103219 · Cited by 1
Variant pathogenicity prediction based on the ESGMM algorithm
Liu, J., Zhang, C., Liu, X., Zhang, Y., Liu, J., & Guan, X. (2024) · Discover Computing · doi:10.1007/s10791-024-09487-w · Cited by 1
Self-citationInter-chain residue-residue distances prediction method for protein complexes using deep learning
Yilin, P. U., Wang, S., Xia, Y., & ZHANG, G. (2024) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0296 · Cited by 1
Gou, W., Tan, Y., Liu, C., Li, M., Fan, G., & Yu, H. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c00856 · Cited by 1
Zhang, Y., Zhong, S., Xu, S., Wang, Z., Xin, C., Ni, F., Yan, F., Lu, X., Sun, S., Wang, H., et al. (2025) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2025.147637 · Cited by 1
Zhen, F., Yu, H., Guan, X., Yue, Z., Gu, L., Li, K., & Zhou, X. (2025) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-025-00788-2 · Cited by 1
Feng, X., Li, H., Wang, Z., Li, N., Liu, Z., Dai, Q., Zhao, L., Hu, L., & Mo, H. (2026) · Future Foods · doi:10.1016/j.fufo.2026.100930 · Cited by 1
Deep Learning Meets Synthetic Biology: Applications Across Molecular, Cellular, and Metabolic Scales
Zhao, Y., Yang, C., Xue, B., Liu, H., Wang, S., & Su, H. (2026) · Green Carbon · doi:10.1016/j.greenca.2025.11.003 · Cited by 2
AI-driven computational methods and benchmarking for T-cell antigen identification
Deng, Y., Que, J., Xue, G., Cai, Y., Yang, W., Wang, Y., Yi, H., Wang, Z., Lin, Y., Zhou, W., et al. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag123 · Cited by 1
Lee, W.-H., Dumontet, L., Jung, K., Lee, H. J., Thapa, G., Oh, T., & Kang, M. (2026) · Applied and Environmental Microbiology · doi:10.1128/aem.01586-25 · Cited by 2
Bisset, B., Kainkaryam, V., Marhamati, S. H., Moradi, A., Amin, R., & Houman, J. (2026) · Journal of Medical Internet Research · doi:10.2196/81314 · Cited by 1
Qian, Z., Zhou, Y., Lin, R., Hong, Y., Xu, Y., Li, S., Gao, J., Zhu, Q., & Zheng, H. (2026) · Journal of Artificial Intelligence Research and Innovation · doi:10.29328/journal.jairi.1001022 · Cited by 1
Application of Protein Language Models to low-N Engineering of metalloproteinase inhibitors
Buxton, E. K., Patel, R., Toumaian, M. R., & Raeeszadeh‐Sarmazdeh, M. (2021) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci54926.2021.00048 · Cited by 1
Hu, S. J., Zeng, J., Lin, X., Zhong, X., Liang, F., Chen, P., & Bai, X. (2025) · International Symposium on Computer Applications and Information Systems (ISCAIS) · doi:10.1109/iscait64916.2025.11010470 · Cited by 1
Ashraf, F. B., Madrigal, V. A., & Lonardi, S. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c00973 · Cited by 1
A Review of Protein Structure Prediction using Deep Learning
Susanty, M., Rajab, T. L. E., & Hertadi, R. (2021) · BIO Web of Conferences · doi:10.1051/bioconf/20214104003 · Cited by 1
Lee, D., Yang, J., & Kim, S. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.12.30.472333 · Cited by 1
Disentanglement of Entropy and Coevolution using Spectral RegularizationPreprint
Wang, H., Feng, S., Liu, S., & Ovchinnikov, S. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.03.04.483009 · Cited by 1
Zhu, Y., Zhang, C., Yu, D., & Zhang, Y. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.07.07.499156 · Cited by 1
Co-evolution integrated deep learning framework for variants generation and fitness predictionPreprint
Tan, X. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.28.526023 · Cited by 1
The power and limits of predicting exon-exon interactions using protein 3D structuresPreprint
Liebold, J., Moral‐Morales, A. D., Manalastas-Cantos, K., Tsoy, O. V., Kurtz, S., Baumbach, J., & Newaz, K. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.03.01.582917 · Cited by 1
NeuroFold: A Multimodal Approach to Generating Novel Protein Variants in silicoPreprint
Amani, K., Fish, M., Smith, M. D., & Castroverde, C. D. M. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.03.12.584504 · Cited by 1
Protein Language Models in Directed EvolutionPreprint
Maguire, R. G., Bloznelyte, K., Adepoju, F., Armean-Jones, M., Dewan, S., Fozzard, S., Goddard, S. E., Gupta, A. K., Ibrahimi, E., Jones, F. P., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.20.608752 · Cited by 1
TooT-PLM-P2S: Incorporating Secondary Structure Information into Protein Language ModelsPreprint
Ghazikhani, H., & Butler, G. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.13.607781 · Cited by 1
Chen, R., Palpant, N. J., Foley, G., & Mikael Bodén, A. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.03.12.642937 · Cited by 1
Mechanism-Aware Inductive Bias Enhances Generalization in Protein-Protein Interaction PredictionPreprint
Deng, S., Wan, X., Mu, Z., Huang, S., & Yan, C. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.07.04.663157 · Cited by 1
Separating selection from mutation in antibody language modelsPreprint
Matsen, F. A., IV, Dumm, W., Sung, K., Johnson, M. M., Rich, D., Starr, T. N., Song, Y. S., Fukuyama, J., & Haddox, H. K. (2026) · eLife · doi:10.7554/elife.109644 · Cited by 3
Teddy: neural inference of epidemiological parameters from viral sequencesPreprint
Garot, V., Blassel, L., Nesterenko, L., Zhukova, A., Alizon, S., & Jacob, L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.05.697728 · Cited by 2
Kabir, M. W. U., Dey, A., Nafees, F., & Hoque, M. T. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.22.701204 · Cited by 2
Tasmin, M., Mohanty, S., Kulkarni, S. G., Farhat, M. R., & Green, A. G. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.30.702134 · Cited by 1
Integrated pretraining with evolutionary information to improve RNA secondary structure predictionPreprint
Yan, Z., Hamilton, W. L., & Blanchette, M. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.01.27.478113
AN ANALYSIS OF PROTEIN STRUCTURE PREDICTION WITH HELP OF ARTIFICIAL INTELLIGENCE
Bhadoriya, A. (2022) · International Journal of Engineering Applied Sciences and Technology · doi:10.33564/ijeast.2022.v07i04.027
Predicting Protein Interaction Sites Using PITHIA
Hosseini, S., & Ilie, L. (2023) · Methods in molecular biology · doi:10.1007/978-1-0716-3327-4_29
VaxSeer: Selecting influenza vaccines through evolutionary and antigenicity modelsPreprint
Shi, W., Wohlwend, J., Wu, M., & Barzilay, R. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.14.567037
Genotype sampling for deep-learning assisted experimental mapping of fitness landscapesPreprint
Wagner, A. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.01.18.576262
Komp, E., Phillips, C., Alanzi, H. N., Zorman, M. L., & Beck, D. A. C. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.02.06.579188
Protein language model-embedded geometric graphs power inter-protein contact prediction
Si, Y., & Yan, C. (2024) · eLife · doi:10.7554/elife.92184.3
Deep Learning Reaction Network: a machine learning framework for modeling time resolved dataPreprint
Alagna, N., Dúzs, B., Koeppl, H., Walther, A., & Gerber, S. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.07.31.606055
Boosting Protein Language Models with Negative Sample Mining
Xu, Y., Zhao, X., Song, X., Wang, B., & Yu, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70381-2_13
AI-enabled alkaline-resistant evolution of protein to apply in mass production
Kang, L., Wu, B., Zhou, B., Tan, P., Kang, Y. (., Yan, Y., Zong, Y., Li, S., Liu, Z., & Hong, L. (2025) · eLife · doi:10.7554/elife.102788.3
Zhao, C., & Wang, S. (2024) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3698587.3701400
RESEARCH ON DETECTION OF SPARTINA ALTERNIFLORA BASED ON SA-YOLO
Chunqing, W. A. N. G., Shang, S., Wang, R., Yang, Z., He, X., & Yue, D. (2025) · INMATEH Agricultural Engineering · doi:10.35633/inmateh-75-38
Almalki, B., & Liao, L. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26094270
Transformer-Based Multi-label Protein Subcellular Localization Prediction
Zhao, W., Zhong, Y., Cao, Y., Zhao, Y., He, W., Chen, Y., & Cheng, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0036-9_33
Bisset, B., Kainkaryam, V., Marhamati, S. H., Moradi, A., Amin, R., & Houman, J. (2025) · doi:10.2196/preprints.81314
Zhou, J., & Wang, C. (2025) · Algorithms · doi:10.3390/a18080465
A Deep Learning Framework for Protein-to-Metal Binding Prediction Using Protein Language Models
Shishir, F. S., Sarker, B., Rahman, F., & Shomaji, S. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2025.3595446
Feng, Z., Zhang, X. M., Wang, H., Hong, X., Zhan, J., & Zhou, Y. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.12.669850 · Cited by 1
Asim, M. N., Ahmed, S., & Dengel, A. R. (2025) · Springer eBooks · doi:10.1007/978-3-031-90454-7_2
Protein Language Model–Aligned Spectra Embeddings for De Novo Peptide SequencingPreprint
Naderializadeh, N., Dallago, C., Soderblom, E. J., & Soderling, S. H. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.01.679857
PEPE: scalable extraction of multi-modal protein language model representations
Zhong, J., Cardente, N., Sandve, G. K., Bashour, H., Abbate, M. F., & Greiff, V. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag375
Separating selection from mutation in antibody language models
Matsen, F. A., IV, Dumm, W., Sung, K., Johnson, M. M., Rich, D., Starr, T. N., Song, Y. S., Fukuyama, J., & Haddox, H. K. (2026) · eLife · doi:10.7554/elife.109644.3 · Cited by 1
Metadata
- Authors
- Rao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A.
- Affiliations
- Facility for Antiproton and Ion Research (DE)
- Meta (United States) (US)
- University of California, Berkeley (US)
- Corresponding author
- Rao, Roshan
- Rives, Alexander
- ORCID
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2021-02-13
- OpenAlex
- W3133458480
- Cited by
- 350
- References
- 47
- Citations by year
- 2026: 63 · 2025: 90 · 2024: 74 · 2023: 45 · 2022: 47 · 2021: 28
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- protein language models, multiple sequence alignment, masked language modeling, protein structure prediction, parameter efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
