SESNet: sequence-structure feature-integrated deep learning method for data-efficient protein engineering
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Deep learning has been widely used for protein engineering. However, it is limited by the lack of sufficient experimental data to train an accurate model for predicting the functional fitness of high-order mutants. Here, we develop SESNet, a supervised deep-learning model to predict the fitness for protein mutants by leveraging both sequence and structure information, and exploiting attention mechanism. Our model integrates local evolutionary context from homologous sequences, the global evolutionary context encoding rich semantic from the universal protein sequence space and the structure information accounting for the microenvironment around each residue in a protein. We show that SESNet outperforms state-of-the-art models for predicting the sequence-function relationship on 26 deep mutational scanning datasets. More importantly, we propose a data augmentation strategy by leveraging the data from unsupervised models to pre-train our model. After that, our model can achieve strikingly high accuracy in prediction of the fitness of protein mutants, especially for the higher order variants (> 4 mutation sites), when finetuned by using only a small number of experimental mutation data (< 50). The strategy proposed is of great practical value as the required experimental effort, i.e., producing a few tens of experimental mutation data on a given protein, is generally affordable by an ordinary biochemical group and can be applied on almost any protein.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkab1061 · Cited by 7,850
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549 · cited 1×
“Thanks to the dramatic breakthrough in deep learning-based technique for predicting protein structure [19, 20], especially AlphaFold 2, it is now possible to efficiently predict protein structures from sequences at a large scale [21].”
BackgroundRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 11×
“We compared our supervised model against the existing state-of-the-art supervised models, ECNet [17], ESM-1b [6]; and unsupervised models, ESM-1v [9], ESM-IF1[23] and MSA transformer[15].”
BackgroundMethodInfluentialHH-suite3 for fast remote homology detection and deep protein annotation
Steinegger, M., Meier, M., Mirdita, M., Vöhringer, H., Haunsberger, S. J., & Söding, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3019-7 · Cited by 1,145
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131 · cited 3×
“…from evolution-related sequences[15, 16]; the second one (global encoder) captures the sequence feature in global protein sequence universe[6, 12]; and the third one (structure module) captures local structural feature around each residue learned from 3D geometric structure of the…”
BackgroundMethodProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009
Disease variant prediction with deep generative models of evolutionary data
Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J. K., Brock, K., Gal, Y., & Marks, D. S. (2021) · Nature · doi:10.1038/s41586-021-04043-8 · Cited by 791 · cited 2×
“Several models, including ESMMSA-1b [37], DeepSequence [14], EVE [38] and the Potts model [27], such as EVmutation [16] and ECNet [39], utilize multiple sequence alignment (MSA) to dig the constraints of evolutionary process in the residues level.”
Method
Research works printed but not linked: 30
- Arnold FH (1998) Design by directed evolution. Acc Chem Res 31(3):125–131 · DOI · Paper page
- Wu Z et al (2019) Machine learning-assisted directed protein evolution with combinatorial libraries. Proc Natl Acad Sci 116(18):8852–8858 · DOI · Paper page
- Cui Y et al (2021) Computational redesign of a PETase for plastic biodegradation under ambient condition by the GRAPE strategy. ACS Catal 11(3):1340–1350 · DOI · Paper page
- Hie BL, Yang KK, Kim PS (2022) Evolutionary velocity with protein language models predicts evolutionary dynamics of diverse proteins. Cell Syst. https://doi.org/10.1016/j.cels.2022.01.003 · DOI · Paper page
- Rao R et al (2019) Evaluating protein transfer learning with TAPE. Adv Neural Inf Process Syst 32:9689
- Meier J et al (2021) Language models enable zero-shot prediction of the effects of mutations on protein function. bioRxiv. https://doi.org/10.1101/2021.07.09.450648 · DOI · Paper page
- Elnaggar A et al (2020) ProtTrans: towards cracking the language of Life’s code through self-supervised deep learning and high performance computing. arXiv Preprint. https://doi.org/10.48550/arXiv.2007.06225 · arXiv · doi:10.48550/arxiv.2007.06225 · Paper page
- Rao RM et al (2021) MSA transformer, in proceedings of the 38th international conference on machine learning. In: Marina M, Tong Z (eds). PMLR: proceedings of machine learning research. p. 8844-8856.
- Luo Y et al (2021) ECNet is an evolutionary context-integrated deep learning framework for protein engineering. Nat Commun 12(1):5743 · DOI · Paper page
- Zhang Z et al (2022) Protein representation learning by geometric structure pretraining. arXiv Preprint. https://doi.org/10.48550/arXiv.2203.06125 · arXiv · doi:10.48550/arxiv.2203.06125 · Paper page
- Hsu C et al (2022) Learning inverse folding from millions of predicted structures. bioRxiv. https://doi.org/10.1101/2022.04.10.487779 · DOI · Paper page
- Lu H et al (2022) Machine learning-aided engineering of hydrolases for PET depolymerization. Nature 604(7907):662–667 · DOI · Paper page
- Wang Z et al (2022) LM-GVP: an extensible sequence and structure informed deep learning framework for protein property prediction. Sci Rep 12(1):6832 · DOI · Paper page
- Gelman S et al (2021) Neural networks to learn protein sequence–function relationships from deep mutational scanning data. Proc Natl Acad Sci 118(48):e2104878118 · DOI · Paper page
- Shroff R et al (2020) Discovery of novel gain-of-function mutations guided by structure-based deep learning. ACS Synth Biol 9(11):2927–2935 · DOI · Paper page
- Zimmer M (2002) Green fluorescent protein (GFP): applications, structure, and related photophysical behavior. Chem Rev 102(3):759–782 · DOI · Paper page
- Melamed D et al (2013) Deep mutational scanning of an RRM domain of the Saccharomyces cerevisiae poly (A)-binding protein. RNA 19(11):1537–1551 · DOI · Paper page
- Minot M, Reddy ST (2022) Nucleotide augmentation for machine learning-guided protein engineering. bioRxiv. https://doi.org/10.1101/2022.03.08.483422 · doi:10.1101/2022.03.08.483422
- Hsu C et al (2022) Learning protein fitness models from evolutionary and assay-labeled data. Nat Biotechnol 40(7):1114–1122 · DOI · Paper page
- Aakre CD et al (2015) Evolving new protein-protein interaction specificity through promiscuous intermediates. Cell 163(3):594–606 · DOI · Paper page
- Sinai S et al (2021) Generative AAV capsid diversification by latent interpolation. bioRxiv. https://doi.org/10.1101/2021.04.16.440236 · DOI · Paper page
- Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579
- Rao R et al (2021) Msa transformer. In international conference on machine learning. PMLR.
- Luo Y et al (2021) ECNet is an evolutionary context-integrated deep learning framework for protein engineering. Nat Commun 12(1):1–14 · DOI · Paper page
- Seemayer S, Gruber M, Söding J (2014) CCMpred—fast and precise prediction of protein residue–residue contacts from correlated mutations. Bioinformatics 30(21):3128–3130 · DOI · Paper page
- Meier J et al (2021) Language models enable zero-shot prediction of the effects of mutations on protein function. Adv Neural Inf Process Syst 34:29287–29303
- Ba JL, Kiros JR, Hinton GE (2016) Layer normalization. arXiv Preprint. https://doi.org/10.48550/arXiv.1607.06450 · arXiv · doi:10.48550/arxiv.1607.06450 · Paper page
- Fukushima K (1975) Cognitron: a self-organizing multilayered neural network. Biol Cybern 20(3):121–136 · DOI · Paper page
- Rollins NJ et al (2019) Inferring protein 3D structure from deep mutation scans. Nat Genet 51(7):1170–1176 · DOI · Paper page
- Bolognesi B et al (2019) The mutational landscape of a prion-like domain. Nat Commun 10(1):1–12 · DOI · Paper page
Cited by
An end-to-end framework for the prediction of protein structure and fitness from single sequence
Chen, Y., Xu, Y., Liu, D., Xing, Y., & Gong, H. (2024) · Nature Communications · doi:10.1038/s41467-024-51776-x · Cited by 45
Xu, Y., Liu, D., & Gong, H. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00716-2 · Cited by 41
AI-enabled alkaline-resistant evolution of protein to apply in mass production
Kang, L., Wu, B., Zhou, B., Tan, P., Kang, Y. (., Yan, Y., Zong, Y., Li, S., Liu, Z., & Hong, L. (2025) · eLife · doi:10.7554/elife.102788 · Cited by 7
Self-citationDesigning diverse and high-performance proteins with a large language model in the loop
Gomez-Uribe, C. A., Gado, J., & Islamov, M. (2025) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1013119 · Cited by 6
Flattening the curve—How to get better results with small deep‐mutational‐scanning datasets
Wirnsberger, G., Pritišanac, I., Oberdorfer, G., & Gruber, K. (2024) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26686 · Cited by 3
Metadata
- Authors
- Li, M., Kang, L., Xiong, Y., Wang, Y. G., Fan, G., Tan, P., & Hong, L.
- Venue
- Journal of Cheminformatics
- Published
- 2023-02-03
- Volume
- 15
- Issue
- 1
- Pages
- 12-12
- ISSN
- 1758-2946 (ISSN-L)
- OpenAlex
- W4319078047
- Cited by
- 35
- References
- 20
- FWCI
- 2.54
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- National Natural Science Foundation of China (12104295, 31630002, 61872094, 61832019)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SESNet: sequence-structure feature-integrated deep learning method for data-efficient protein engineering. https://app.cochat.ai/papers/10.1186/s13321-023-00688-x
- The paper
- Li, M., Kang, L., Xiong, Y., Wang, Y. G., Fan, G., Tan, P., & Hong, L. (2023). SESNet: sequence-structure feature-integrated deep learning method for data-efficient protein engineering. Journal of Cheminformatics. https://doi.org/10.1186/s13321-023-00688-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.