Protein language models learn evolutionary statistics of interacting sequence motifs
Open accessPublished version of a preprint
The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Protein language models (pLMs) have emerged as potent tools for predicting and designing protein structure and function, and the degree to which these models fundamentally understand the inherent biophysics of protein structure stands as an open question. Motivated by a finding that pLM-based structure predictors erroneously predict nonphysical structures for protein isoforms, we investigated the nature of sequence context needed for contact predictions in the pLM Evolutionary Scale Modeling (ESM-2). We demonstrate by use of a “categorical Jacobian” calculation that ESM-2 stores statistics of coevolving residues, analogously to simpler modeling approaches like Markov Random Fields and Multivariate Gaussian models. We further investigated how ESM-2 “stores” information needed to predict contacts by comparing sequence masking strategies, and found that providing local windows of sequence information allowed ESM-2 to best recover predicted contacts. This suggests that pLMs predict contacts by storing motifs of pairwise contacts. Our investigation highlights the limitations of current pLMs and underscores the importance of understanding the underlying mechanisms of these models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig. 2. Deep learning structure-based methods predict isoforms as fragments of full-length structures with exposed aggregation-prone residues. (A) Top: alignment between human myoglobin (UniProt: P02144) and human isoform Q8WVH6 (UniProt: Q8WVH6). Left: X-ray structure of human myoglobin [Protein Data Bank (PDB): 3RGK] shown with the heme cofactor, and the two missing helices A and B in the isoform colored in gray. Right: predicted structures of isoform Q8WVH6 of Myoglobin from AlphaFold2, OmegaFold, and ESMFold have low root-mean-squared deviation (RMSD) to structure 3RGK (0.49, 1.01, and 0.81 Å respectively). Bottom: surfaces of protein fragments corresponding to the isoform sequence. The isoform structure models all have exposed hydrophobic residues corresponding to where helices A and B reside in the full-length structure (indicated with an arrow), quantified here using the spatial aggregation propensity (SAP) score (16). Structure predictions of isoforms from Human Prostaglandin E synthase 3 (B), Human Caspase-9 (C), and Human Nfs1 cysteine desulfurase (D) all share similar trends, where the isoform structure model contains a significant patch of residues with high SAP score. In (B–D), the structure model depicted is from ESMFold. (E) For 18 isoforms previously identified in the literature as isoforms where splicing events occur in structured domains, we calculated RMSD to a reference structure of the full-length protein, and the change in average SAP for the isoform fragment in comparison to the sequence aligned in the full-length protein. We found that for AlphaFold2, OmegaFold, and ESMFold, isoform structure models generally had low RMSD to the reference structure, predicted with relatively high pLDDT, along with increased average SAP.](https://figures.cochat.ai/paper-figures/c01ccd8ae5674e38be65c4416a7c1e7f/protein-language-models-learn-evolutionary-statistics-of-interacting-sequence-figure-2.jpg)

![Fig. 4. Categorical Jacobian is an unsupervised method to extract coevolutionary signal and uniformly evaluate any pLM. (A) Scheme of the categorical Jacobian calculation. Each residue in a sequence of length L is changed to A different types of amino acids, where A is the size of the alphabet (for proteins, A=20). By computing how the output changes with respect to the input, a matrix of size [L, A, L, A] is obtained. (B) This categorial Jacobian allows for comparing a nonlinear method like ESM-2 and a simple linear method, exemplified here for large ribosomal subunit protein RL29 (UniProt: P0A7M7). We can compare the coevolutionary weights obtained from a linear model, calculated using inverse covariance, and the categorical Jacobian calculated from ESM-2. (C) Contacts calculated from the categorical Jacobian from ESM-2 outperform the inverse covariance calculation from ref. 25 (Average long-range P@L/2 of 0.67 vs. 0.80, respectively). (D) Comparing contact accuracy from the categorical Jacobian and the supervised contact prediction head (Average long-range P@L/2 of 0.80 vs. 0.87, respectively). (E) Correlation between covariation parameters from linear model and ESM-2 Jacobians increase with model size. Top: Distribution of Spearman correlation coefficients between contacts from linear model and ESM-2 Jacobians. Bottom: average Spearman R, varying σ cutoff for linear model values close to zero. For (C–E), N = 1,431 proteins.](https://figures.cochat.ai/paper-figures/ad02f4b012a24748997166172511258f/protein-language-models-learn-evolutionary-statistics-of-interacting-sequence-figure-4.jpg)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We observed that the model uses information from one end to specify the other end across a distance of ~1 to 2 kb, reflecting the model’s understanding of the tight evolutionary linkage of the two terminal elements”
From Sequence modeling and design from molecular to genome scale with Evo (2024) · cites this paper as [70]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391
Self-citationEvolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
Self-citationTM-align: a protein structure alignment algorithm based on the TM-score
Zhang, Y. (2005) · Nucleic Acids Research · doi:10.1093/nar/gki524 · Cited by 3,774
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280
Cited by
Genome modelling and design across all domains of life with Evo 2
Brixi, G., Durrant, M. G., Ku, J., Naghipourfar, M., Poli, M., Sun, G., Brockman, G., Chang, D., Fanton, A., Gonzalez, G. A., et al. (2026) · Nature · doi:10.1038/s41586-026-10176-5 · Cited by 702
Self-citationSequence modeling and design from molecular to genome scale with Evo
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. (2024) · Science · doi:10.1126/science.ado9336 · Cited by 580
Self-citationTransformers and genome language models
Consens, M. E., Dufault, C., Wainberg, M., Forster, D., Karimzadeh, M., Goodarzi, H., Theis, F. J., Moses, A. M., & Wang, B. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01007-9 · Cited by 196
InterPLM: discovering interpretable features in protein language models via sparse autoencoders
Simon, E. P., & Zou, J. (2025) · Nature Methods · doi:10.1038/s41592-025-02836-7 · Cited by 169
Biophysics-based protein language models for protein engineering
Gelman, S., Johnson, B., Freschlin, C. R., Sharma, A., D’Costa, S., Peters, J., Gitter, A., & Romero, P. A. (2025) · Nature Methods · doi:10.1038/s41592-025-02776-2 · Cited by 69
Evaluating the representational power of pre-trained DNA language models for regulatory genomics
Tang, Z., Somia, N., Yu, Y., & Koo, P. K. (2025) · Genome biology · doi:10.1186/s13059-025-03674-8 · Cited by 85
Protein structure-informed bacteriophage genome annotation with Phold
Bouras, G., Grigson, S. R., Mirdita, M., Heinzinger, M., Papudeshi, B., Mallawaarachchi, V., Green, R., Kim, R. S., Mihalia, V., Psaltis, A. J., et al. (2026) · Nucleic Acids Research · doi:10.1093/nar/gkaf1448 · Cited by 71
PLM-interact: extending protein language models to predict protein-protein interactions
Liu, D., Young, F., Lamb, K. D., Quiros, A. C., Pancheva, A., Miller, C. J., Macdonald, C., Robertson, D. L., & Yuan, K. (2025) · Nature Communications · doi:10.1038/s41467-025-64512-w · Cited by 72
The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modelingPreprint
Cornman, A., West-Roberts, J. A., Camargo, A. P., Roux, S., Beracochea, M., Mirdita, M., Ovchinnikov, S., & Hwang, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.14.607850 · Cited by 73
Self-citationConditional language models enable the efficient design of proficient enzymesPreprint
Munsamy, G., Illanes-Vicioso, R., Funcillo, S., Nakou, I. T., Lindner, S., Ayres, G., Sheehan, L. S., Moss, S., Eckhard, U., Lorenz, P., et al. (2024) · bioRxiv · doi:10.1101/2024.05.03.592223 · Cited by 55
Gujral, O. S., Bafna, M., Alm, E. J., & Berger, B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2506316122 · Cited by 87
Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models
Li, F.-Z., Amini, A. P., Yue, Y., Yang, K. K., & Lu, A. X. (2024) · doi:10.1101/2024.02.05.578959 · Cited by 51
Protein Language Models: Is Scaling Necessary?
Fournier, Q., Vernon, R. M., van der Sloot, A., Schulz, B., Chandar, S., & Langmead, C. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.09.23.614603 · Cited by 26
Adams, E., Bai, L., Lee, M., Yu, Y.-Y., & Alquraishi, M. K. S. (2025) · bioRxiv · doi:10.1101/2025.02.06.636901 · Cited by 65
Nucleotide dependency analysis of genomic language models detects functional elements
da Silva, P. T., Karollus, A., Hingerl, J., Galindez, G., Wagner, N., Hernandez‐Alias, X., Incarnato, D., & Gagneur, J. (2025) · Nature Genetics · doi:10.1038/s41588-025-02347-3 · Cited by 41
Emerging frontiers in protein structure prediction following the AlphaFold revolution
Rennie, M. L., & Oliver, M. R. (2025) · Journal of The Royal Society Interface · doi:10.1098/rsif.2024.0886 · Cited by 40
Protein Language Models: Applications and Perspectives
Leclercq, M., & Droit, A. (2025) · Journal of Proteome Research · doi:10.1021/acs.jproteome.5c00506 · Cited by 43
Predicting genome-wide functional constraints with GPN-Star
Ye, C., Benegas, G., Albors, C., Li, J. C., Prillo, S., Fields, P. D., Clarke, B., & Song, Y. S. (2026) · Nature · doi:10.1038/s41586-026-11005-5 · Cited by 34
Mehdiabadi, M., Conte, A. D., Nugnes, M. V., Aspromonte, M. C., Tosatto, S. C. E., & Piovesan, D. (2025) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.70045 · Cited by 32
Artificial intelligence methods for protein folding and design
Zhang, Z., Ou, C., Cho, Y., Akiyama, Y., & Ovchinnikov, S. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103066 · Cited by 31
Self-citationProtein language models trained on biophysical dynamics inform mutation effects
Hou, C., Zhao, H., & Shen, Y. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2530466123 · Cited by 30
Prediction of antibiotic resistance mechanisms using a protein language model
Yagimoto, K., Hosoda, S., Sato, M., & Hamada, M. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae550 · Cited by 25
Training bias and sequence alignments shape protein–peptide docking by AlphaFold and related methods
Guan, L., & Keating, A. E. (2025) · Protein Science · doi:10.1002/pro.70331 · Cited by 26
Lamb, K. D., Hughes, J., Lytras, S., Young, F., Koci, O., Herzig, J. C., Lovell, S. C., Grove, J., Yuan, K., & Robertson, D. L. (2026) · Nature Communications · doi:10.1038/s41467-026-69569-9 · Cited by 24
Evaluating generalization in protein–ligand cofolding methods
Škrinjar, P., Eberhardt, J., Studer, G., Tauriello, G., Schwede, T., & Durairaj, J. (2026) · Nature Structural & Molecular Biology · doi:10.1038/s41594-026-01797-5 · Cited by 22
Orthrus: toward evolutionary and functional RNA foundation models
Fradkin, P., Shi, R. “., Dalal, T., Isaev, K., Frey, B. J., Lee, L. J., Morris, Q., & Wang, B. (2026) · Nature Methods · doi:10.1038/s41592-026-03064-3 · Cited by 22
DRfold2 is a deep learning-based tool that enables efficient and accurate RNA structure prediction
Li, Y., Feng, C., Zhang, X., Tsukiyama, S., Feng, D., & Zhang, Y. (2026) · PLoS Biology · doi:10.1371/journal.pbio.3003659 · Cited by 24
ProtNote: a multimodal method for protein–function annotation
Char, S., Corley, N., Alamdari, S., Yang, K., & Amini, A. P. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf170 · Cited by 20
Nayar, G., Tartici, A., & Altman, R. B. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013424 · Cited by 21
Scaling down protein language modeling with MSA PairformerPreprint
Akiyama, Y., Zhang, Z., Mirdita, M., Steinegger, M., & Ovchinnikov, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.02.668173 · Cited by 19
Self-citationE1: Retrieval-Augmented Protein Encoder ModelsPreprint
Jain, S., Beazer, J., Ruffolo, J. A., Bhatnagar, A., & Madani, A. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.12.688125 · Cited by 22
Advancing protein structure prediction beyond AlphaFold2
Park, S., Myung, S., & Baek, M. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.102985 · Cited by 19
Towards Mechanistic Interpretability of Graph Transformers via Attention GraphsPreprint
El, B., Choudhury, D., Lió, P., & Joshi, C. K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.12352 · Cited by 17
Karagöl, A., Karagöl, T., Li, M., & Zhang, S. (2024) · Pharmaceutical Research · doi:10.1007/s11095-024-03786-z · Cited by 15
Evaluating variant effect prediction across virusesPreprint
Gurev, S. F., Youssef, N., Jain, N., Mehrotra, A., Leung, S. R. M., Jackson, A., & Marks, D. S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.04.668549 · Cited by 17
FLIP2: Expanding Protein Fitness Landscape Benchmarks for Real-World Machine Learning ApplicationsPreprint
Didi, K., Alamdari, S., Lu, A. X., Wittmann, B. J., Johnston, K. E., Amini, A. A., Madani, A., Czeneszew, M., Dallago, C., & Yang, K. K. (2026) · bioRxiv · doi:10.64898/2026.02.23.707496 · Cited by 16
Laine, É., & Freiberger, M. I. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2024.102979 · Cited by 14
Zhai, J., Qi, X., Cai, L., Liu, Y., Tang, H., Xie, L., & Wang, J. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf212 · Cited by 15
Single-Sequence, Structure Free Allosteric Residue Prediction with Protein Language ModelsPreprint
Kannan, G. R., Hie, B., & Kim, P. S. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.03.616547 · Cited by 16
Towards Interpretable Protein Structure Prediction with Sparse AutoencodersPreprint
Parsan, N., Yang, D. J., & Yang, J. J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.08764 · Cited by 16
Evaluating Pretrained Protein Language Model Embeddings as Proxies for Functional Similarity
Shaw, R., Love, S. D., & McWhite, C. D. (2025) · Journal of Molecular Evolution · doi:10.1007/s00239-025-10282-4 · Cited by 13
Intrinsic dataset features drive mutational effect prediction by protein language modelsPreprint
Vieira, L. C., Lin, S., & Wilke, C. O. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.08.710389 · Cited by 14
Leveraging multi-modal feature learning for predictions of antibody viscosity
Anapindi, K. D. B., Liu, K., Wang, W., Yao, Y. Q., He, Y., Hsieh, E. J., Huang, Y., & Tomazela, D. M. (2025) · mAbs · doi:10.1080/19420862.2025.2490788 · Cited by 14
Predictive and therapeutic applications of protein language models.
Furui, K., Sakano, K., & Ohue, M. (2025) · Allergology International · doi:10.1016/j.alit.2025.08.004 · Cited by 10
Feldman, J., Brogi, M., & Skolnick, J. (2026) · Computational and Structural Biotechnology Journal · doi:10.34133/csbj.0142 · Cited by 14
Wang, Y., Wang, Z., Sadeh, G., Zancato, L., Achille, A., Karypis, G., & Rangwala, H. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.29.620988 · Cited by 19
Isolation of structural constraints by ensemble-conditioned protein sequence designPreprint
Shuai, R. W., Zhou, L., Nesr, G. E., Lin, J., Lu, T., Mathews, I. I., Bhatti, S., Yuan, J., Kouba, P., & Huang, P. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.30.679633 · Cited by 12
Multi-state Protein Design with DynamicMPNN
Abrudan, A., Ojeda, S. P., Joshi, C. K., Greenig, M., Engelberger, F., Khmelinskaia, A., Meiler, J., Vendruscolo, M., & Knowles, T. P. J. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2507.21938 · Cited by 16
A survey of downstream applications of evolutionary scale modeling protein language models
Yang, Q., Yu, J., & Zheng, J. (2025) · Quantitative Biology · doi:10.1002/qub2.70013 · Cited by 14
Subramanian, A. M., Martinez, Z. A., Lourenço, A. L., Yuan, S. C., Liu, S., Wang, T.-Y., Chou, T., & Thomson, M. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.12.22.573145 · Cited by 11
The DNA dialect: a comprehensive guide to pretrained genomic language models
Veiner, M., & Supek, F. (2026) · Molecular Systems Biology · doi:10.1038/s44320-025-00184-4 · Cited by 13
Probing the dark energy in the functional protein universe
Galpern, E. A., Bueno, C., Sánchez, I. E., Wolynes, P., & Ferreiro, D. (2025) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2531111123 · Cited by 9
Huang, J., Ge, L., Wu, Y. J., Tian, Y., Wu, J., Gao, Q., Pan, L., Meng, S., Zhang, H., & Qin, Z. (2026) · Nucleic Acids Research · doi:10.1093/nar/gkag656 · Cited by 11
LassoESM a tailored language model for enhanced lasso peptide property prediction
Mi, X., Barrett, S. E., Mitchell, D. A., & Shukla, D. (2025) · Nature Communications · doi:10.1038/s41467-025-63412-3 · Cited by 9
Scaling SMILES-Based Chemical Language Models for Therapeutic Peptide Engineering
Feller, A. L., Secor, M., Swanson, S., Wilke, C. O., & Deibler, K. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c00652 · Cited by 12
Ko, Y. S., Parkinson, J., & Wang, W. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag014 · Cited by 9
Toward life with a 19–amino acid alphabet through generative artificial intelligence design
Liu, L., Rochereau, C., Kozlov, S., Urtecho, G., Liu, X., Zhao, J., Wang, J., Huang, Y., Qu, Y., Zhang, Z., et al. (2026) · Science · doi:10.1126/science.aeb5171 · Cited by 9
Self-citationKoehl, A., Prillo, S., Liu, M., Xiong, J. T., Weng, L., Savage, D. F., & Song, Y. S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.19.706898 · Cited by 9
Protein design and optimization for synthetic cells
Frohn, B. P., Kohyama, S., & Schwille, P. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00318-1 · Cited by 8
Protein evolution as a complex system
Gall, B., Pulsford, S. B., Matthews, D. S., Spence, M. A., Kaczmarski, J. A., Chen, J. Z., Sandhu, M., Stone, E. A., Nichols, J., & Jackson, C. J. (2025) · Nature Chemical Biology · doi:10.1038/s41589-025-01977-2 · Cited by 7
Integrative modeling meets deep learning: Recent advances in modeling protein assemblies
Shor, B., & Schneidman‐Duhovny, D. (2024) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2024.102841 · Cited by 7
Interpretable machine learning reveals a diverse arsenal of anti-defenses in bacterial virusesPreprint
Lopatina, A., Ferenc, M., Bartlau, N., Wolfram, M., Steensen, K., Muscia, S., Hussain, F. A., Schnurer, M., Afjehi‐Sadat, L., Pollak, S., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.14.598830 · Cited by 7
Gener anno : A Genomic Foundation Model for Metagenomic AnnotationPreprint
Li, Q., Wu, W., Zhu, Y., Feng, F., Ye, J., & Wang, Z. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.04.656517 · Cited by 20
Nainani, J., Reimer, B. M., Watts, C., Jensen, D., & Green, A. G. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.08.22.671739 · Cited by 11
Risk‑Based Prediction of Novel AMR Variants Using Protein Language ModelsPreprint
Wood, J. J., Portelli, S., Ascher, D. B., & Furnham, N. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.12.672331 · Cited by 7
Uncovering the Mechanistic Landscape of Regulatory DNA with Deep LearningPreprint
Seitz, E. E., McCandlish, D. M., Kinney, J. B., & Koo, P. K. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.07.681052 · Cited by 6
Imam, I. A., Bailey, S., Wang, D., Zeng, S., Xu, D. H., & Shao, Q. (2024) · Langmuir · doi:10.1021/acs.langmuir.4c04140 · Cited by 5
Erkanli, M. E., Jang, Y., Malli, A., El‐Halabi, K., Ryu, C., & Kim, J. R. (2025) · ACS Synthetic Biology · doi:10.1021/acssynbio.5c00257 · Cited by 5
Molecular Modeling and machine learning for predicting high-concentration antibody viscosity
Baizhigitova, D., Wu, I.-E., Kalejaye, L. A., & Lai, P. (2026) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2026.115839 · Cited by 5
Explainable Deep Multilevel Attention Learning for Predicting Protein Carbonylation Sites
Zhang, J., Qian, J., Wang, P., Liu, X., Zhang, F., Chai, H., & Zou, Q. (2025) · Advanced Science · doi:10.1002/advs.202500581 · Cited by 6
Bushuiev, A., Bushuiev, R., Pimenova, O., Zadorozhny, N., Samusevich, R., Manaskova, E., Kim, R. S., Stärk, H., Sedlář, J., Steinegger, M., et al. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2411.02109 · Cited by 6
Liu, T., Huang, T., Jin, W., Chu, T., Ying, R., & Zhao, H. (2026) · Genome Research · doi:10.1101/gr.281001.125 · Cited by 7
Kolchina, A., Dubanevics, I., Kondrashov, F. A., & Kalinina, O. V. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.17.706292 · Cited by 6
Generative Artificial Intelligence for Function-Driven De Novo Enzyme Design
Xuan, Q., De-hang, W., Shi, Z., Xiaoping, L., & Ma, H. (2025) · Synthetic biology and engineering · doi:10.70322/sbe.2025.10015 · Cited by 4
Phage evolutionary relationships emerge from protein language model-based proteome representation
Panigrahi, S., Ansaldi, M., & Ginet, N. (2025) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaf134 · Cited by 4
Black-box data: a new paradigm for biomedicine in the AI era
Naef, L., & Bronstein, M. M. (2026) · Chemical Science · doi:10.1039/d6sc01189f · Cited by 4
Srivastava, A. K. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103128 · Cited by 4
Protein Language Models Capture Structural and Functional Epistasis in a Zero-Shot SettingPreprint
Nambiar, A., Littlefield, S. B., Cuellar, C., Khorana, R., & Maslov, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.09.14.676130 · Cited by 4
AlphaFold reveals but sometimes distorts an organizational principle of protein foldingPreprint
Clore, M. F., Thole, J. F., Dontha, S., Sharma, P., Greenberg, N., Strub, M., Starich, M. R., Ott, C. M., Jensen, D. R., Volkman, B. F., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.30.697132 · Cited by 5
Bouras, G. S., Lim, S. W., Durr, L., Vreugde, S., Goesmann, A., Edwards, R. A., & Schwengers, O. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.31.715528 · Cited by 5
Coevolutionary mining of prokaryotic non-coding elements with a genome language modelPreprint
Li, D., Brixi, G., Kim, A., Fiamenghi, M. B., Driscoll, C. L., Evans, S. A., Gao, L., Ivanova, N., Kyrpides, N. C., Deisseroth, K. A., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.22.753630 · Cited by 4
Self-citationFold-Switching Proteins Push the Boundaries of Conformational Ensemble Prediction
Lee, M., & Porter, L. L. (2026) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092524-114822 · Cited by 4
Deep learning on protein language model embeddings unlocks accurate prediction of protein solubility
Cui, J., Jiang, Y., Wang, J., Jiang, H., Fang, J., Jiang, J., Chen, Q., Zhong, S., & Wang, X. (2026) · Frontiers in Microbiology · doi:10.3389/fmicb.2026.1716930 · Cited by 3
Cucco, B. (2026) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.6c00026 · Cited by 3
SPPIDER-seq: sequence-based partner-aware predictor of protein-protein interaction sites
Porollo, A., Jadhav, O., Alvarez, A., & Chen, J. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag388 · Cited by 3
Vinod, R., Amini, A. P., Crawford, L. A., & Yang, K. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013925 · Cited by 3
Learning gene interactions and functional landscapes from entire bacterial proteomesPreprint
Sethi, P., Chevrette, M. G., & Zhou, J. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.03.19.644232 · Cited by 7
Detecting and quantifying overparametrization in RNA language models with REDIALPreprint
Teng, D., Qiu, Y., Sakthivel, G., Aranganathan, A., Herron, L., & Tiwary, P. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.11.724344 · Cited by 3
BOTANIC-1: a series of long-context plant genomic foundation models in the agentic eraPreprint
Barozet, A., Cabeli, V., du Terrail, J. O., Rukhovich, A., Janssoone, T., Klajer, G., Sheikhitarghi, Z., Andrews, G., Véran, C., & Strouk, L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.04.749355 · Cited by 4
Higinbotham, H. R., Arbour, C. A., & Imperiali, B. (2025) · Protein Science · doi:10.1002/pro.70352 · Cited by 2
A Survey of Pretrained Protein Language Models
Pokharel, S., Pratyush, P., Chaudhari, M., Heinzinger, M., Caragea, D., Saigo, H., & KC, D. B. (2025) · Methods in molecular biology · doi:10.1007/978-1-0716-4623-6_1 · Cited by 4
Physics Determines and Evolution Shapes Basic Units of Protein Function and Allosteric Regulation
Berezovsky, I. N. (2026) · Methods in molecular biology · doi:10.1007/978-1-0716-4836-0_7 · Cited by 2
Evaluating transformer-based models for structural characterization of orphan proteins
Seçkin, E., Colinet, D., Danchin, É., & Sarti, E. (2026) · Bioinformatics Advances · doi:10.1093/bioadv/vbag215 · Cited by 2
Unsupervised discovery of functional sequence patterns from protein language model with MotifAE
Hou, C., Лю, Д., & Shen, Y. (2026) · Nature Communications · doi:10.1038/s41467-026-77333-2 · Cited by 2
Seq2Pocket: augmenting protein language models for spatially consistent binding site prediction
Škrhák, V., Polák, L., Novotný, M., & Hoksza, D. (2026) · Bioinformatics · doi:10.1093/bioinformatics/btag778 · Cited by 2
Xu, G., Lv, Y., Zhang, R., Xia, X., Wang, Q., & Ma, J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.17.629067 · Cited by 2
Liu, T., Huang, T., Jin, W., Chu, T., Ying, R., & Zhao, H. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.22.649977 · Cited by 3
Pretrained protein language models choose between sequence novelty and structural completenessPreprint
Subramanian, A. M., Martinez, Z. A., & Thomson, M. W. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.01.679905 · Cited by 3
Zhang, Z., Akiyama, Y., Cho, Y., Jajoo, S., & Ovchinnikov, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.16.688745 · Cited by 3
Self-citationCalibrated Variant Effect Prediction at the Residue Level Using Conditional Score DistributionsPreprint
Passi, G., Amittai, S., & Schneidman-Duhovny, D. (2025) · bioRxiv · doi:10.1101/2025.11.24.690189 · Cited by 2
Metadata
- Authors
- Zhang, Z., Wayment-Steele, H. K., Brixi, G., Wang, H., Kern, D., & Ovchinnikov, S.
- Affiliations
- Harvard University (US)
- Howard Hughes Medical Institute (US)
- Corresponding author
- Ovchinnikov, Sergey
- ORCID
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2024-10-28
- Volume
- 121
- Issue
- 45
- Pages
- e2406285121-e2406285121
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC11551344
- OpenAlex
- W4403824354
- Cited by
- 140
- References
- 39
- Citations by year
- 2026: 66 · 2025: 56 · 2024: 16
- FWCI
- 18.79
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- Jane Coffin Childs (N/A)
- Howard Hughes Medical Institute (N/A)
- National Institutes of Health (DP5OD026389)
- National Foundation of Science (MCB2032259)
- Swiss National Science Foundation (205321_192371)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein language models, ESM-2, contact prediction, sequence motifs, protein isoforms, sequence masking, protein structure prediction, model interpretability
- MeSH terms
- Markov Chains, Models, Molecular, Protein Conformation, Proteins, Evolution, Molecular, Amino Acid Motifs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.