Pairing interacting protein sequences using masked language modeling
Open accessPublished version of a preprint
Abstract
Predicting which proteins interact together from amino acid sequences is an important task. We develop a method to pair interacting protein sequences which leverages the power of protein language models trained on multiple sequence alignments (MSAs), such as MSA Transformer and the EvoFormer module of AlphaFold. We formulate the problem of pairing interacting partners among the paralogs of two protein families in a differentiable way. We introduce a method called Differentiable Pairing using Alignment-based Language Models (DiffPALM) that solves it by exploiting the ability of MSA Transformer to fill in masked amino acids in multiple sequence alignments using the surrounding context. MSA Transformer encodes coevolution between functionally or structurally coupled amino acids within protein chains. It also captures inter-chain coevolution, despite being trained on single-chain data. Relying on MSA Transformer without fine-tuning, DiffPALM outperforms existing coevolution-based pairing methods on difficult benchmarks of shallow multiple sequence alignments extracted from ubiquitous prokaryotic protein datasets. It also outperforms an alternative method based on a state-of-the-art protein language model trained on single sequences. Paired alignments of interacting protein sequences are a crucial ingredient of supervised deep learning methods to predict the three-dimensional structure of protein complexes. Starting from sequences paired by DiffPALM substantially improves the structure prediction of some eukaryotic protein complexes by AlphaFold-Multimer. It also achieves competitive performance with using orthology-based pairing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391 · see citing works
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Protein complex prediction with AlphaFold-Multimer
Evans, R., O’Neill, M., Pritzel, A., Antropova, N., Senior, A., Green, T., Žídek, A., Bates, R., Blackwell, S., Yim, J., et al. (2021) · doi:10.1101/2021.10.04.463034 · Cited by 3,808 · see citing works
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434 · see citing works
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2205.14135 · Cited by 1,214 · cited 1× · see citing works
“Using e.g. FlashAttention [69], or using a simpler MSA-based protein language model in place of MSA Transformer, might help alleviate some of these limitations.”
MethodProtein 3D Structure Computed from Evolutionary Sequence Variation
Marks, D. S., Colwell, L. J., Sheridan, R., Hopf, T. A., Pagnani, A., Zecchina, R., & Sander, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028766 · Cited by 1,144 · see citing works
Improved prediction of protein-protein interactions using AlphaFold2
Bryant, P., Pozzati, G., & Elofsson, A. (2021) · doi:10.1101/2021.09.15.460468 · Cited by 1,025 · see citing works
Single-sequence protein structure prediction using language models from deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., & AlQuraishi, M. (2021) · doi:10.1101/2021.08.02.454840 · Cited by 455 · see citing works
Single-sequence protein structure prediction using a language model and deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01432-w · Cited by 455 · see citing works
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341 · see citing works
Tubiana, J., Schneidman-Duhovny, D., & Wolfson, H. J. (2022) · Nature Methods · doi:10.1038/s41592-022-01490-7 · Cited by 274 · see citing works
The binary protein-protein interaction landscape of Escherichia coli
Rajagopala, S. V., Sikorski, P., Kumar, A., Mosca, R., Vlasblom, J., Arnold, R., Franca-Koh, J., Pakala, S. B., Phanse, S., Ceol, A., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.2831 · Cited by 233 · see citing works
Protein sequence design by conformational landscape optimization
Norn, C., Wicky, B. I. M., Juergens, D., Liu, S., Kim, D., Tischer, D., Koepnick, B., Anishchenko, I., Players, F., Baker, D., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2017228118 · Cited by 144 · see citing works
Correlated mutations in models of protein sequences: phylogenetic and structural effects
Lapedes, A. S., Giraud, B., Liu, L., & Stormo, G. D. (1999) · Institute of Mathematical Statistics Lecture Notes - Monograph Series · doi:10.1214/lnms/1215455556 · Cited by 124 · see citing works
Green, A. G., Elhabashy, H., Brock, K. P., Maddamsetti, R., Kohlbacher, O., & Marks, D. S. (2021) · Nature Communications · doi:10.1038/s41467-021-21636-z · Cited by 114 · see citing works
Inter-residue, inter-protein and inter-family coevolution: bridging the scales
Szurmant, H., & Weigt, M. (2018) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2017.10.014 · Cited by 81 · see citing works
Improved Docking of Protein Models by a Combination of Alphafold2 and ClusPro
Ghani, U., Desta, I., Jindal, A., Khan, O., Jones, G., Hashemi, N., Kotelnikov, S., Padhorny, D., Vajda, S., & Kozakov, D. (2021) · doi:10.1101/2021.09.07.459290 · Cited by 68 · see citing works
Improved Multimer Prediction using Massive Sampling with AlphaFold in CASP15
Wallner, B. (2023) · doi:10.22541/au.168170986.60158084/v1 · Cited by 53 · see citing works
Limits and potential of combined folding and docking
Pozzati, G., Zhu, W., Bassot, C., Lamb, J., Kundrotas, P., & Elofsson, A. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btab760 · Cited by 25 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Genomic language model predicts protein co-regulation and function
Hwang, Y., Cornman, A. L., Kellogg, E. H., Ovchinnikov, S., & Girguis, P. R. (2024) · Nature Communications · doi:10.1038/s41467-024-46947-9 · Cited by 99 · see citing works
“In cases where multiple interacting pairs are found withinanorganism(e.g. histidinekinases(HK)andresponseregulators (RR)), prediction of interacting pairs is particularly dif fi cult 48 .”
- Authors
- Lupo, U., Sgarbossa, D., & Bitbol, A.-F.
- Affiliations
- SIB Swiss Institute of Bioinformatics (CH)
- Corresponding author
- Lupo, Umberto
- Sgarbossa, Damiano
- Bitbol, Anne-Florence
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2024-06-24
- Volume
- 121
- Issue
- 27
- Pages
- e2311887121-e2311887121
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC11228504
- OpenAlex
- W4399940408
- Cited by
- 31
- References
- 20
- Citations by year
- 2026: 13 · 2025: 11 · 2024: 7
- FWCI
- 4.22
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (cc-by)
- Funders
- DeepMind
- European Commission (851173, HORIZON2020)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- protein-protein interaction prediction, masked language modeling, multiple sequence alignment, coevolution, paralogs, protein language models, protein complex structure prediction, AlphaFold-Multimer
- MeSH terms
- Algorithms, Amino Acid Sequence, Proteins, Sequence Alignment, Computational Biology, Sequence Analysis, Protein, Databases, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
