Embedding-based alignment: combining protein language models with dynamic programming alignment to detect structural similarities in the twilight-zone
Open accessPublished version of a preprint
Abstract
Motivation: Language models are routinely used for text classification and generative tasks. Recently, the same architectures were applied to protein sequences, unlocking powerful new approaches in the bioinformatics field. Protein language models (pLMs) generate high-dimensional embeddings on a per-residue level and encode a “semantic meaning” of each individual amino acid in the context of the full protein sequence. These representations have been used as a starting point for downstream learning tasks and, more recently, for identifying distant homologous relationships between proteins. Results: In this work, we introduce a new method that generates embedding-based protein sequence alignments (EBA) and show how these capture structural similarities even in the twilight zone, outperforming both classical methods as well as other approaches based on pLMs. The method shows excellent accuracy despite the absence of training and parameter optimization. We demonstrate that the combination of pLMs with alignment methods is a valuable approach for the detection of relationships between proteins in the twilight-zone. Availability and implementation: The code to run EBA and reproduce the analysis described in this article is available at: https://git.scicore.unibas.ch/schwede/EBA and https://git.scicore.unibas.ch/schwede/eba_benchmark.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkab1061 · Cited by 7,850 · see citing works
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 2× · see citing works
“Benchmarks have been performed with three pre-trained state of the art pLMs: ProstT5 (Heinzinger et al. 2023), ProtT5-XL-UniRef50 (ProtT5) (Elnaggar et al. 2022), and esm1b_t33_650M_UR50S (ESM-1b) (Rives et al. 2021).”
Potter, S. C., Luciani, A., Eddy, S. R., Park, Y., Lopez, R., & Finn, R. D. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky448 · Cited by 2,607 · see citing works
Fast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2022) · doi:10.1101/2022.02.07.479398 · Cited by 2,598 · see citing works
Fast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01773-0 · Cited by 2,596 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · see citing works
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · cited 3× · see citing works
“Two examples of embedding-based alignment (EBA) methods were introduced by Bepler and Berger (2021) and Hamamsy et al. (2023).”
Flagged first, then by citations
- Authors
- Pantolini, L., Studer, G., Pereira, J., Durairaj, J., Tauriello, G., & Schwede, T.
- Affiliations
- SIB Swiss Institute of Bioinformatics (CH)
- Corresponding author
- Schwede, Torsten
- ORCID
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2024-01-01
- Volume
- 40
- Issue
- 1
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC10792726
- OpenAlex
- W4390590489
- Cited by
- 55
- References
- 10
- Citations by year
- 2026: 17 · 2025: 20 · 2024: 12 · 2023: 6
- FWCI
- 4.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- edoc (University of Basel) (acceptedVersion)
- Funders
- SIB Swiss Institute of Bioinformatics
- Biozentrum, University of Basel
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Keywords
- protein language models, protein sequence alignment, structural similarity, twilight zone, distant homology detection, dynamic programming alignment, protein embeddings, protein structure comparison
- MeSH terms
- Amino Acid Sequence, Amino Acids, Language, Proteins, Sequence Alignment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
