CoChat
Sign Up
PreprintCited by 221Protein Structure and DynamicsMachine Learning in BioinformaticsEnzyme Structure and Function

Learning protein sequence embeddings using information from structure

Open access
Bepler, Tristan, Berger, Bonnie·2019·arXiv
DOI 10.48550/arxiv.1902.08661arXiv 1902.08661PMC PMC12947628
Open this paper in CoChat — free account

Abstract

Inferring the structural properties of a protein from its amino acid sequence is a challenging yet important problem in biology. Structures are not known for the vast majority of protein sequences, but structure is critical for understanding function. Existing approaches for detecting structural similarity between proteins from sequence are unable to recognize and exploit structural patterns when sequences have diverged too far, limiting our ability to transfer knowledge between structurally related proteins. We newly approach this problem through the lens of representation learning. We introduce a framework that maps any protein sequence to a sequence of vector embeddings --- one per amino acid position --- that encode structural information. We train bidirectional long short-term memory (LSTM) models on protein sequences with a two-part feedback mechanism that incorporates information from (i) global structural similarity between proteins and (ii) pairwise residue contact maps for individual proteins. To enable learning from structural similarity information, we define a novel similarity measure between arbitrary-length sequences of vector embeddings based on a soft symmetric alignment (SSA) between them. Our method is able to learn useful position-specific embeddings despite lacking direct observations of position-level correspondence between sequences. We show empirically that our multi-task framework outperforms other sequence-based methods and even a top-performing structure-based alignment method when predicting structural similarity, our goal. Finally, we demonstrate that our learned embeddings can be transferred to other protein sequence problems, improving the state-of-the-art in transmembrane domain prediction.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1902.08661

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The multi-task learning approach outperforms a two-step learning approach (SSA-LSTM) for structural similarity prediction.

    “Furthermore, the multi-task learning approach outperforms a two-step learning approach presented previously (SSA-LSTM)27.”

    From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]

  • Learned protein representations improve predictive ability for downstream prediction problems through transfer learning.

    “Learned protein representations improve predictive ability for downstream prediction problems through transfer learning (Figure 5a).”

    From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]

  • Language models learn summary representations that can be used to capture sequence relationships and link sequence to function via transfer learning.

    “They also learn summary representations, powerful features that can be used to better capture sequence relationships and link sequence to function via transfer learning 27,28,32-35.”

    From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]

  • Representations learned by language models are powerful features for learning of variant fitness as a subsequent supervised learning task.

    “New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”

    From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]

  • The sequence representations (vector embeddings) learned by the language model are powerful features for solving other prediction problems.

    “The key insight was that the sequence representations (vector embeddings) learned by the language model were powerful features for solving other prediction problems.”

    From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Cited by

Sorted by citations

Related

Authors
Bepler, T., & Berger, B.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
Volume
2019
Pages
3162-3178
DOI
10.48550/arxiv.1902.08661
arXiv
1902.08661
PMC
PMC12947628
OpenAlex
W2909727437
Cited by
219
References
0
Citations by year
2026: 4 · 2025: 29 · 2024: 25 · 2023: 29 · 2022: 47 · 2021: 46 · 2020: 33 · 2019: 6
FWCI
100.20
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Institute of General Medical Sciences (R01GM081871)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
Keywords
protein sequence embeddings, protein structural similarity, sequence similarity measures, bidirectional LSTM, multi-task learning, transmembrane domain prediction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review