LM-GVP: an extensible sequence and structure informed deep learning framework for protein property prediction
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Proteins perform many essential functions in biological systems and can be successfully developed as bio-therapeutics. It is invaluable to be able to predict their properties based on a proposed sequence and structure. In this study, we developed a novel generalizable deep learning framework, LM-GVP, composed of a protein Language Model (LM) and Graph Neural Network (GNN) to leverage information from both 1D amino acid sequences and 3D structures of proteins. Our approach outperformed the state-of-the-art protein LMs on a variety of property prediction tasks including fluorescence, protease stability, and protein functions from Gene Ontology (GO). We also illustrated insights into how a GNN prediction head can inform the fine-tuning of protein LMs to better leverage structural information. We envision that our deep learning framework will be generalizable to many protein property prediction problems to greatly accelerate protein engineering and drug development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“With the recent advances in large pretrained language models (LMs) from natural languages 8–10 , various types of LMs have also been adapted to protein modeling by treating protein sequences as the language of life, where tokens are AAs.”
MethodUMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101
Biopython: freely available python tools for computational molecular biology and bioinformatics
Cock, P. J. A., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., Friedberg, I., Hamelryck, T., Kauff, F., Wilczynski, B., et al. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp163 · Cited by 6,080
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549 · cited 1×
“Recent breakthroughs in deep learning approaches including AlphaFold 2,3 , trRosetta 4 and a three-track deep neural net approach by Baek et al. 5 , have exceeded performance based on traditional approaches for modeling the folding processes.”
MethodUniversal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 1×
“In the training phase, we adopted the gradual unfreezing technique developed by Howard and Ruder 38 by first learn the parameters in the GVP network while keeping the parameters in the LM frozen until converge.”
MethodThe Gene Ontology resource: enriching a GOld mine
The Gene Ontology Consortium, Carbon, S., Douglass, E., Good, B. M., Unni, D. R., Harris, N. L., Mungall, C. J., Basu, S., Chisholm, R. L., Dodson, R. J., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1113 · Cited by 3,555
Improved protein structure prediction using potentials from deep learning
Senior, A. W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., Žídek, A., Nelson, A. W. R., Bridgland, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1923-7 · Cited by 3,378
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Research works printed but not linked: 18
- Waudby, C. A., Dobson, C. M. & Christodoulou, J. Nature and regulation of protein folding on the ribosome. Trends Biochem. Sci. 44, 914–926 (2019). · DOI · Paper page
- Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. Proc. Natl. Acad. Sci. 117, 1496 (2020). · DOI · Paper page
- Hamley, I. W. The amyloid beta peptide: A chemist’s perspective. Role in Alzheimer’s and fibrillization. Chem. Rev. 112, 5147–5192 (2012). · DOI · Paper page
- Elnaggar, A. et al. ProtTrans: Towards cracking the language of Life’s code through self-supervised learning. bioRxiv 2020.07.12.199554 (2021). https://doi.org/10.1101/2020.07.12.199554. · DOI · Paper page
- Meier, J. et al. Language models enable zero-shot prediction of the effects of mutations on protein function. bioRxiv 2021.07.09.450648 (2021) https://doi.org/10.1101/2021.07.09.450648. · DOI · Paper page
- Villegas-Morcillo, A. et al. Unsupervised protein embeddings outperform hand-crafted sequence and structure features at predicting molecular function. Bioinformatics 37, 162–170 (2021). · DOI · Paper page
- Gligorijević, V. et al. Structure-based protein function prediction using graph convolutional networks. Nat. Commun. 12, 3168 (2021). · DOI · Paper page
- Ingraham, J., Garg, V., Barzilay, R. & Jaakkola, T. Generative Models for Graph-Based Protein Design. in Advances in Neural Information Processing Systems (eds. Wallach, H. et al.) vol. 32 (Curran Associates, Inc., 2019).
- Ashburner, M. et al. Gene Ontology: Tool for the unification of biology. Nat. Genet. 25, 25–29 (2000). · DOI · Paper page
- McGeagh, J. D., Ranaghan, K. E. & Mulholland, A. J. Protein dynamics and enzyme catalysis: Insights from simulations. Protein Dyn. Exp. Comput. Approaches 1814, 1077–1092 (2011).
- Doshi, U. & Hamelberg, D. The Dilemma of Conformational Dynamics in Enzyme Catalysis: Perspectives from Theory and Experiment. in Protein Conformational Dynamics (eds. Han, K., Zhang, X. & Yang, M.) 221–243 (Springer International Publishing, 2014). https://doi.org/10.1007/978-3-319-02970-2_10. · DOI
- Ester, M., Kriegel, H.-P., Sander, J. & Xu, X. A Density-Based Algorithm for Discovering clusters in large spatial databases with noise. in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining 226–231 (AAAI Press, 1996).
- Yasunaga, M., Ren, H., Bosselut, A., Liang, P. & Leskovec, J. QA-GNN: Reasoning with language models and knowledge graphs for question answering. (2021). · DOI · Paper page
- Berman, H. M. et al. The protein data bank. Nucleic Acids Res. 28, 235–242 (2000). · DOI · Paper page
- Kuhlman, B. et al. Design of a novel globular protein fold with atomic-level accuracy. Science 302, 1364 (2003). · DOI · Paper page
- Khatib, F. et al. Algorithm discovery by protein folding game players. Proc. Natl. Acad. Sci. 108, 18949 (2011). · DOI · Paper page
- Yang, J., Roy, A. & Zhang, Y. BioLiP: A semi-manually curated database for biologically relevant ligand–protein interactions. Nucleic Acids Res. 41, D1096–D1103 (2013). · DOI · Paper page
- Pedregosa, F. et al. Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Cited by
Convolutions are competitive with transformers for protein sequence pretraining
Yang, K. K., Fusi, N., & Lu, A. X. (2024) · Cell Systems · doi:10.1016/j.cels.2024.01.008 · Cited by 137
Masked inverse folding with sequence transfer for protein representation learning
(2022) · doi:10.1093/protein/gzad015 · Cited by 110
GENA-LM: A Family of Open-Source Foundational DNA Language Models for Long Sequences
Fishman, V., Kuratov, Y., Shmelev, A., Petrov, M., Penzar, D., Shepelin, D., Chekanov, N., Kardymon, O., & Burtsev, M. (2023) · doi:10.1101/2023.06.12.544594 · Cited by 137
“For human data, we provide five evaluation results obtained on each of the human dataset folds.”
Wang, T., Xiang, G., He, S., Su, L., Wang, Y., Yan, X., & Lu, H. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae409 · Cited by 57
Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models
Li, F.-Z., Amini, A. P., Yue, Y., Yang, K. K., & Lu, A. X. (2024) · doi:10.1101/2024.02.05.578959 · Cited by 51
“Other tasks use structure as an additional input or target, but they generally make only modest improvements on function prediction tasks (Mansoor et al., 2021; Wang et al., 2022; Yang et al., 2023; Su et al., 2023).”
Background
Metadata
- Authors
- Wang, Z., Combs, S. A., Brand, R., Calvo, M. R., Xu, P., Price, G., Golovach, N., Salawu, E. O., Wise, C. J., Ponnapalli, S. P., & Clark, P. M.
- Venue
- Scientific Reports
- Published
- 2022-04-27
- Volume
- 12
- Issue
- 1
- Pages
- 6832-6832
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W4224939843
- Cited by
- 85
- References
- 21
- FWCI
- 5.11
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: LM-GVP: an extensible sequence and structure informed deep learning framework for protein property prediction. https://app.cochat.ai/papers/10.1038/s41598-022-10775-y
- The paper
- Wang, Z., Combs, S. A., Brand, R., Calvo, M. R., Xu, P., Price, G., Golovach, N., Salawu, E. O., Wise, C. J., Ponnapalli, S. P., & Clark, P. M. (2022). LM-GVP: an extensible sequence and structure informed deep learning framework for protein property prediction. Scientific Reports. https://doi.org/10.1038/s41598-022-10775-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.