The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Learning protein sequence embeddings using information from structure
Open accessAbstract
Inferring the structural properties of a protein from its amino acid sequence is a challenging yet important problem in biology. Structures are not known for the vast majority of protein sequences, but structure is critical for understanding function. Existing approaches for detecting structural similarity between proteins from sequence are unable to recognize and exploit structural patterns when sequences have diverged too far, limiting our ability to transfer knowledge between structurally related proteins. We newly approach this problem through the lens of representation learning. We introduce a framework that maps any protein sequence to a sequence of vector embeddings --- one per amino acid position --- that encode structural information. We train bidirectional long short-term memory (LSTM) models on protein sequences with a two-part feedback mechanism that incorporates information from (i) global structural similarity between proteins and (ii) pairwise residue contact maps for individual proteins. To enable learning from structural similarity information, we define a novel similarity measure between arbitrary-length sequences of vector embeddings based on a soft symmetric alignment (SSA) between them. Our method is able to learn useful position-specific embeddings despite lacking direct observations of position-level correspondence between sequences. We show empirically that our multi-task framework outperforms other sequence-based methods and even a top-performing structure-based alignment method when predicting structural similarity, our goal. Finally, we demonstrate that our learned embeddings can be transferred to other protein sequence problems, improving the state-of-the-art in transmembrane domain prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, the multi-task learning approach outperforms a two-step learning approach presented previously (SSA-LSTM)27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]
In the citing paper
“Learned protein representations improve predictive ability for downstream prediction problems through transfer learning (Figure 5a).”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]
In the citing paper
“They also learn summary representations, powerful features that can be used to better capture sequence relationships and link sequence to function via transfer learning 27,28,32-35.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]
In the citing paper
“New work has since shown that the representations learned by language models are also powerful features for learning of variant fitness as a subsequent supervised learning task 32,34, building on earlier observations that language models can improve protein property prediction through transfer learning 27.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]
In the citing paper
“The key insight was that the sequence representations (vector embeddings) learned by the language model were powerful features for solving other prediction problems.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
Items 201–221 of 221
DenseHMM: Learning Hidden Markov Models by Learning Dense Representations
Sicking, J., Pintz, M., Akila, M., & Wirtz, T. (2022) · International Conference on Pattern Recognition Applications and Methods · doi:10.5220/0010821800003122 · Cited by 2
Protein matchmaking through representation learning
Heinzinger, M., Dallago, C., & Rost, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.09.007
Horne, R. I., Possenti, A., Chia, S., Brotzakis, Z. F., Staats, R., Nowinska, M., & Vendruscolo, M. (2022) · Research Square · doi:10.21203/rs.3.rs-1236129/v1
Hossain, M. N., Islam, M. M., & Islam, M. K. B. (2021) · International Conference on Science & Contemporary Technologies (ICSCT) · doi:10.1109/icsct53883.2021.9642535
Koehl, P. (2021) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.739876259.793585293
Pre-training Protein Models with Molecular Dynamics Simulations for Drug BindingPreprint
Wu, F., Zhang, Q., Radev, D., Wang, Y., Jin, X., Jiang, Y., Li, S. Z., & Niu, Z. (2022) · Research Square · doi:10.21203/rs.3.rs-1566483/v1 · Cited by 2
Melanoma Detection by Analysing Mutations in Gene DNA Sequences and Their Primary Protein Structures
Peiris, P., Jayaweera, J. A. A. M., & Ganegoda, G. U. (2022) · International Conference on Image Processing and Robotics (ICIPRoB) · doi:10.1109/iciprob54042.2022.9798719
Li, S., Tian, T., Zhang, Z., Zou, Z., Zhao, D., & Zeng, J. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4161090
Boomerang-shaped neural embeddings for NK landscapes
Santana, R., Liefooghe, A., & Derbel, B. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528856
Mir, R. A., Shafi, S. M., & Zargar, S. M. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-99045-5.00006-9
Biomedical Knowledge Representation Learning
Zeng, Z., Liu, Z., Lin, Y., & Sun, M. (2023) · Springer eBooks · doi:10.1007/978-981-99-1600-9_12
VIEPred: Predicting Viral Immune Evasion with Multimodal Representation Learning
Liang, B., Wang, N., Huang, H., Hou, Y., & Qiang, X. (2024) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2868/1/012029 · Cited by 1
Hybrid Model Design For Protein Function Prediction
Zhao, S., Zhang, X., Zhao, Z., Qian, P., Liu, W., Zeng, Z., Veeravalli, B., Dai, L., Nordlund, P., Prabhu, N., et al. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10781799
Li, Y. (2024) · Theoretical and Natural Science · doi:10.54254/2753-8818/2024.la18884
Saha, D., Borah, N. J., & Mamud, A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-31550-3.00014-7
Kernelized Fuzzy System for Predicting Therapeutic Peptides via Deep Stacked Encoder
Guo, X., Ding, Y., & Zou, Q. (2025) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2025.3558344
A descriptor-free machine learning framework to improve antigen discovery for bacterial pathogens
Podda, M., Savojardo, C., Martelli, P. L., Casadio, R., Sîrbu, A., Priami, C., & Brozzi, A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0323895
Predicting Viral Escape: A Contrastive Learning Framework with Lightweight Protein Language ModelsPreprint
Tenekeci, S., Sezgin, E., & Tekır, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5383577
Ghosh, S., & Mitra, P. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.28.702421
Metadata
- Authors
- Bepler, T., & Berger, B.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Volume
- 2019
- Pages
- 3162-3178
- arXiv
- 1902.08661
- PMC
- PMC12947628
- OpenAlex
- W2909727437
- Cited by
- 219
- References
- 0
- Citations by year
- 2026: 4 · 2025: 29 · 2024: 25 · 2023: 29 · 2022: 47 · 2021: 46 · 2020: 33 · 2019: 6
- FWCI
- 100.20
- Type
- conference paper
- Language
- en
- Open access
- green
- pmc.ncbi.nlm.nih.gov
- Locations
- PubMed (publishedVersion)
- PubMed Central (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Institute of General Medical Sciences (R01GM081871)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Structural Biology › Enzyme Structure and Function
- Keywords
- protein sequence embeddings, protein structural similarity, sequence similarity measures, bidirectional LSTM, multi-task learning, transmembrane domain prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
