The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Unified rational protein engineering with sequence-based deep representation learning
Open accessPublished version of a preprint
Abstract
Rational protein engineering requires a holistic understanding of protein function. Here, we apply deep learning to unlabeled amino-acid sequences to distill the fundamental features of a protein into a statistical representation that is semantically rich and structurally, evolutionarily and biophysically grounded. We show that the simplest models built on top of this unified representation (UniRep) are broadly applicable and generalize to unseen regions of sequence space. Our data-driven approach predicts the stability of natural and de novo designed proteins, and the quantitative function of molecularly diverse mutants, competitively with the state-of-the-art methods. UniRep further enables two orders of magnitude efficiency improvement in a protein engineering task. UniRep is a versatile summary of fundamental protein features that can be applied across protein engineering informatics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“They also learn summary representations, powerful features that can be used to better capture sequence relationships and link sequence to function via transfer learning 27,28,32-35.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [28]
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [28]
In the citing paper
“In recent years, deep learning architectures that were originally developed for natural language processing (NLP) tasks, such as translating a sentence from one language into another, have been applied successfully to the creation of numerical protein representations from amino acid sequences 17 , 29 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [29]
In the citing paper
“These fingerprints are used together with a 1900-dimensioanl enzyme representation, called UniRep vector 29 , as the input for a gradient boosted decision tree model 34 to predict the K M value for an enzyme-substrate pair.”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We used basic agglomerative clustering (implemented in Python with sklearn 73 ) with the average linkage metric and Euclidean distance for all vector representations, and a precomputed Levenshtein distance matrix using the Python Levenshtein package for the sequence-only control.”
MethodBerman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196 · cited 4×
“We computed the full sequence of hidden states at every position of 3,720 single-domain proteins from the SCOP database 60 (for which we had secondary structure annotation for each of the positions obtained from the PDB 80 ).”
MethodInfluential(2008) · Journal of Machine Learning Research · Cited by 33,051
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 6,122
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
Valiev, M., Bylaska, E. J., Govind, N., Kowalski, K., Straatsma, T. P., van Dam, H. J. J., Wang, D., Nieplocha, J., Aprà, E., Windus, T. L., et al. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2010.04.018 · Cited by 5,849
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 3,827
Molecular dynamics simulations of biomolecules
Karplus, M., & McCammon, J. (2002) · Nature Structural Biology · doi:10.1038/nsb0902-646 · Cited by 3,401
Research works printed but not linked: 8
- Brookes, D. H., Park, H. & Listgarten, J. Conditioning by adaptive sampling for robust design. Proc. Machine Learn. Res. 97, 773–782 (2019).
- Hernández-Lobato, J. M., Requeima, J., Pyzer-Knapp, E. O. & Aspuru-Guzik, A. Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. Preprint at arXiv https://arxiv.org/abs/1706.01825 (2017). · arXiv
- Snoek, J., Larochelle, H. & Adams, R. P. in Advances in Neural Information Processing Systems Vol. 25 (eds. Pereira, F. et al.) 2951–2959 (Curran Associates, Inc., 2012).
- Griffiths, R.-R. & Hernández-Lobato, J. M. Constrained Bayesian optimization for automaticchemical design. Preprint at arXiv https://arxiv.org/abs/1709.05501 (2017). · arXiv
- Yang, K. K., Chen, Y., Lee, A. & Yue, Y. Batched stochastic Bayesian optimization via combinatorial constraints design. Preprint at arXiv https://arxiv.org/abs/1904.08102 (2019). · arXiv
- González, J., Longworth, J., James, D. C. & Lawrence, N. D. Bayesian optimization for synthetic gene design. Preprint at arXiv https://arxiv.org/abs/1505.01627 (2015). · arXiv
- Jouppi, N. P. et al. In-datacenter performance analysis of a tensorprocessing unit. In Proc. 44th Annual International Symposium of Computer Architecture Vol. 45, 1–12 (ACM, 2017).
- Salimans, T. & Kingma, D. P. Weight normalization: a simple reparameterization to accelerate training of deep neural networks. Preprint at arXiv https://arxiv.org/abs/1602.07868 (2016). · arXiv
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
“Alley et al. also made strides in the area of protein analysis, creating statistical summaries that capture key properties of proteins and help neural networks learn with less data 43 .”
BackgroundSignalP 6.0 predicts all five types of signal peptides using protein language models
Teufel, F., Armenteros, J. J. A., Johansen, A. R., Gíslason, M. H., Pihl, S. I., Tsirigos, K. D., Winther, O., Brunak, S., von HEIJNE, G., & Nielsen, H. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01156-3 · Cited by 3,088
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329
A guide to machine learning for biologists
Greener, J. G., Kandathil, S. M., Moffat, L., & Jones, D. T. (2022) · Nature Reviews Molecular Cell Biology · doi:10.1038/s41580-021-00407-0 · Cited by 2,127
“biological sequences Protein engineering [68] Variable input size Long training times”
BackgroundSignalP 6.0 predicts all five types of signal peptides using protein language models
Teufel, F., Armenteros, J. J. A., Johansen, A. R., Gíslason, M. H., Pihl, S. I., Tsirigos, K. D., Winther, O., Brunak, S., von HEIJNE, G., & Nielsen, H. (2022) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000524414 · Cited by 1,905
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
“, 2020), biology (Alley et al., 2019; Rives et al., 2021), and even across multiple modalities (Das et al.”
BackgroundLarge language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01618-2 · Cited by 1,078
Machine-learning-guided directed evolution for protein engineering
Yang, K. K., Wu, Z., & Arnold, F. H. (2019) · Nature Methods · doi:10.1038/s41592-019-0496-6 · Cited by 1,130
ProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009
ProtGPT2 is a deep unsupervised language model for protein design
Ferruz, N., Schmidt, S., & Höcker, B. (2022) · Nature Communications · doi:10.1038/s41467-022-32007-7 · Cited by 795
“Given both the growth of proteinsequence databases and the lack of annotation for a signi fi cant part of the protein space, protein sequences have become great candidates for unsupervised training 4,10,11 and now offer the opportunity to encode and generate protein sequences.”
BackgroundMethodSimulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937
Directed Evolution: Methodologies and Applications
Wang, Y., Xue, P., Cao, M., Yu, T., Lane, S., & Zhao, H. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.1c00260 · Cited by 745
Evaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719
Language models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.07.09.450648 · Cited by 635
Deep learning for molecular design—a review of the state of the art
Elton, D. C., Boukouvalas, Z., Fuge, M. D., & Chung, P. W. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c9me00039a · Cited by 683
Chen, L., Tan, X., Wang, D., Zhong, F., Liu, X., Yang, T., Luo, X., Chen, K., Jiang, H., & Zheng, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa524 · Cited by 607
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483
Efficient evolution of human antibodies from general protein language models
Hie, B., Shanker, V. R., Xu, D., Bruun, T. U. J., Weidenbacher, P. A. B., Tang, S., Wu, W. C. H., Pak, J. E., & Kim, P. S. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01763-2 · Cited by 472
Single-sequence protein structure prediction using a language model and deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01432-w · Cited by 455
Self-citationSingle-sequence protein structure prediction using language models from deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., & AlQuraishi, M. (2021) · doi:10.1101/2021.08.02.454840 · Cited by 455
Self-citationAhdritz, G., Bouatta, N., Floristean, C., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T. J., Berenberg, D., Fisk, I., Zanichelli, N., et al. (2024) · Nature Methods · doi:10.1038/s41592-024-02272-z · Cited by 439
“These include the ESMFold method for protein structure prediction 50 , which replaces MSAs with protein language models 51–53 , and FastFold, a community effort that has implemented substantial improvements including fast model-parallel training and inference 41 .”
Self-citationLearning inverse folding from millions of predicted structuresPreprint
Hsu, C., Verkuil, R., Liu, J. J., Lin, Z., Hie, B., Sercu, T., Lerer, A. K., & Rives, A. W. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.10.487779 · Cited by 492
Genome-wide prediction of disease variant effects with a deep protein language model
Brandes, N., Goldman, G., Wang, C. H., Ye, C., & Ntranos, V. (2023) · Nature Genetics · doi:10.1038/s41588-023-01465-0 · Cited by 489
Low-N protein engineering with data-efficient deep learning
Biswas, S., Khimulya, G., Alley, E. C., Esvelt, K. M., & Church, G. M. (2021) · Nature Methods · doi:10.1038/s41592-021-01100-y · Cited by 422
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 424
Expanding functional protein sequence spaces using generative adversarial networks
Repecka, D., Jauniskis, V., Karpus, L., Rembeza, E., Rokaitis, I., Zrimec, J., Poviloniene, S., Laurynenas, A., Viknander, S., Abuajwa, W., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00310-5 · Cited by 388
High-resolution de novo structure prediction from primary sequence
Wu, R., Ding, F., Wang, R., Shen, R., Zhang, X., Luo, S., Su, C., Wu, Z., Xie, Q., Berger, B., et al. (2022) · doi:10.1101/2022.07.21.500999 · Cited by 392
“Similar to the ESM-1b model ( 12 ), the overall architecture of OmegaPLM is a self-attention model ( 24 ) in which each token is a single amino acid.”
MethodYi, D., Bayer, T., Badenhorst, C. P. S., Wu, S., Doerr, M., Höhne, M., & Bornscheuer, U. T. (2021) · Chemical Society Reviews · doi:10.1039/d0cs01575j · Cited by 387
Mega-scale experimental analysis of protein folding stability in biology and design
Tsuboyama, K., Dauparas, J., Chen, J., Laine, E., Mohseni Behbahani, Y., Weinstein, J. J., Mangan, N. M., Ovchinnikov, S., & Rocklin, G. J. (2023) · Nature · doi:10.1038/s41586-023-06328-6 · Cited by 350
“Large-scale proteolysis data have already had a key role in the development of machine learning methods for protein design and protein biophysics 19,20 .”
BackgroundRao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350
Learning the language of viral evolution and escape
Hie, B., Zhong, E. D., Berger, B., & Bryson, B. (2021) · Science · doi:10.1126/science.abd7331 · Cited by 361
De novo protein design—From new structures to programmable functions
Kortemme, T. (2024) · Cell · doi:10.1016/j.cell.2023.12.028 · Cited by 383
“In principle, function is also encoded in these structures and sequences, and this encoding has been used by machine learning models to generate functional proteins 21,23–25 .”
Enzyme discovery and engineering for sustainable plastic recycling
Zhu, B., Wang, D., & Wei, N. (2022) · Trends in Biotechnology · doi:10.1016/j.tibtech.2021.02.008 · Cited by 366
BERTology Meets Biology: Interpreting Attention in Protein Language Models
Vig, J., Madani, A., Varshney, L. R., Xiong, C., Socher, R., & Rajani, N. F. (2020) · doi:10.1101/2020.06.26.174417 · Cited by 338
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 365
“lutional networks pretrained on 3D-structure contact prediction [13,116], LSTM language models [3], as well as non-deep-”
BackgroundInfluentialTransformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341
“Lastly, Alley et al. (2019) presented a method of fine-tuning where a pretrained language model is further trained on the MSA of the sequence of interest (‘evotuning’).”
BackgroundMethodInfluentialUsing deep learning to annotate the protein universe
Bileschi, M. L., Belanger, D., Bryant, D. H., Sanderson, T., Carter, B., Sculley, D., Bateman, A., DePristo, M. A., & Colwell, L. J. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01179-w · Cited by 332
Protein design and variant prediction using autoregressive generative models
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., & Marks, D. S. (2021) · Nature Communications · doi:10.1038/s41467-021-22732-w · Cited by 320
Machine learning for metabolic engineering: A review
Lawson, C. E., Martí, J. M., Radivojević, T., Jonnalagadda, S. V. R., Gentz, R., Hillson, N. J., Peisert, S., Kim, J., Simmons, B. A., Petzold, C. J., et al. (2020) · Metabolic Engineering · doi:10.1016/j.ymben.2020.10.005 · Cited by 335
AlQuraishi, M. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz422 · Cited by 333
Self-citationDeep diversification of an AAV capsid protein by machine learning
Bryant, D., Bashir, A., Sinai, S., Jain, N. K., Ogden, P. J., Riley, P. F., Church, G. M., Colwell, L. J., & Kelsic, E. D. (2021) · Nature Biotechnology · doi:10.1038/s41587-020-00793-4 · Cited by 319
Self-citationMachine learning for antimicrobial peptide identification and design
Wan, F., Wong, F. J., Collins, J. J., & de la Fuente‐Núñez, C. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00152-x · Cited by 317
“Specifically, amino acid and protein features learned from protein neural language models 39,76,77 trained on large-scale protein sequences have shown strong predictive power on tasks such as predicting protein structure, stability and function after introducing mutations.”
BackgroundChakraborty, C., Bhattacharya, M., Pal, S., & Lee, S. (2023) · Current Research in Biotechnology · doi:10.1016/j.crbiot.2023.100164 · Cited by 311
Machine learning for functional protein design
Notin, P., Rollins, N. J., Gal, Y., Sander, C., & Marks, D. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 304
“This has led to a wide diversity of models, inspired by learning paradigms initially introduced in the natural language-processing literature, such as autoregressive modeling 20,52,61,141 , masked-language modeling 15,146,175,176 or seq2seq architectures 177,178 .”
Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering
Yang, J., Li, F.-Z., & Arnold, F. H. (2024) · ACS Central Science · doi:10.1021/acscentsci.3c01275 · Cited by 268
Elnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 364
Machine learning in protein structure prediction
AlQuraishi, M. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.04.005 · Cited by 263
Self-citationSaProt: Protein Language Modeling with Structure-aware VocabularyPreprint
Su, J., Han, C., Zhou, Y., Shan, J., Zhou, X., & Yuan, F. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.01.560349 · Cited by 264
“Moreover, prominent PLMs like UniRep (Al-ley et al., 2019), ProtTrans (Elnaggar et al., 2021), ESM (Rives et al., 2019; Meier et al., 2021; Rao et al., 2021; Lin et al., 2022), and Evoformer (Hu et al., 2022; Jumper et al., 2021) have showcased outstanding performance across a diverse array of tasks pertaining to prot…”
BackgroundPredictProtein - Predicting Protein Structure and Function for 29 Years
Bernhofer, M., Dallago, C., Karl, T., Satagopam, V., Heinzinger, M., Littmann, M., Olenyi, T., Qiu, J., Schütze, K., Yachdav, G., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab354 · Cited by 246
Recent advances in de novo protein design: Principles, methods, and applications
Pan, X., & Kortemme, T. (2021) · Journal of Biological Chemistry · doi:10.1016/j.jbc.2021.100558 · Cited by 228
“Moreover, the most recent developments of deep learning for protein structure prediction (41–43) foreshadow new methods in the design, taking advantage of learned principles of the protein structure (44, 45).”
BackgroundMethodAccelerating antibiotic discovery through artificial intelligence
Melo, M. C. R., Maasch, J. R. M. A., & de la Fuente-Nunez, C. (2021) · Communications Biology · doi:10.1038/s42003-021-02586-0 · Cited by 234
“The latter led to an embedded representation that could be used to derive a protein ’ s secondary structure, thermal stability, deep mutational scanning classi fi cation, and even the functional impact of mutations 31 .”
BackgroundMethodDeep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227
ProGen: Language Modeling for Protein GenerationPreprint
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi, R. R., Huang, P., & Socher, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.07.982272 · Cited by 383
Fine-tuning protein language models boosts predictions across diverse tasks
Schmirler, R., Heinzinger, M., & Rost, B. (2024) · Nature Communications · doi:10.1038/s41467-024-51844-2 · Cited by 229
Fast and Flexible Protein Design Using Deep Graph Neural Networks
Strokach, A. V., Becerra, D., Corbi‐Verge, C., Perez‐Riba, A., & Kim, P. M. (2020) · Cell Systems · doi:10.1016/j.cels.2020.08.016 · Cited by 230
Rapid in silico directed evolution by a protein language model with EVOLVEpro
Jiang, K., Yan, Z., Bernardo, M. D., Sgrizzi, S. R., Villiger, L., Kayabölen, A., Kim, B., Carscadden, J. K., Hiraizumi, M., Nishimasu, H., et al. (2024) · Science · doi:10.1126/science.adr6006 · Cited by 234
Antibody structure prediction using interpretable deep learning
Ruffolo, J. A., Sulam, J., & Gray, J. J. (2022) · Patterns · doi:10.1016/j.patter.2021.100406 · Cited by 216
“In the recent UniRep method, RNNs were demonstrated to learn rich feature representations from protein sequences when trained on nextamino-acid prediction.(25) For our purposes, we developed an RNN encoder-decoder model(26); the encoder is a bidirectional long short-term memory (biLSTM) and the decoder is a long short…”
MethodInformed training set design enables efficient machine learning-assisted directed protein evolution
Wittmann, B. J., Yue, Y., & Arnold, F. H. (2021) · Cell Systems · doi:10.1016/j.cels.2021.07.008 · Cited by 215
Generating functional protein variants with variational autoencoders
Hawkins‐Hooker, A., Depardieu, F., Baur, S., Couairon, G., Chen, A., & Bikard, D. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008736 · Cited by 217
Machine Learning-Guided Protein Engineering
Kouba, P., Kohout, P., Haddadi, F., Bushuiev, A., Samusevich, R., Sedlář, J., Damborský, J., Pluskal, T., Šivic, J., & Mazurenko, S. (2023) · ACS Catalysis · doi:10.1021/acscatal.3c02743 · Cited by 218
Metabolic Engineering: Methodologies and Applications
Volk, M., Tran, V. G., Tan, S., Mishra, S., Fatma, Z., Boob, A. G., Li, H., Xue, P., Martin, T. A., & Zhao, H. (2022) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00403 · Cited by 217
Learning protein fitness models from evolutionary and assay-labeled data
Hsu, C., Nisonoff, H. M., Fannjiang, C., & Listgarten, J. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01146-5 · Cited by 240
“We assessed a total of 13 published machine learning methods for protein fitness (1) weak-positive only learning methods that rely on probability density modeling of evolutionarily related sequences (profile HMM 27 , EVmutation 23 and DeepSequence 24 ); (2) semi-supervised learning methods that rely on unlabeled UniRe…”
BackgroundMethodInfluentialCan we predict T cell specificity with digital biology and machine learning?
Hudson, D., Fernandes, R. A., Basham, M., Ogg, G. S., & Koohy, H. (2023) · Nature reviews. Immunology · doi:10.1038/s41577-023-00835-3 · Cited by 209
(2023) · doi:10.1101/2023.12.07.570727 · Cited by 207
“Hybrid models combine the respective strengths of family-specific alignment-based and family-agnostic language models, such as the MSA Transformer [Rao et al., 2021], evotuned UniRep [Alley et al., 2019a], Tranception [Notin et al., 2022a] and TranceptEVE [Notin et al., 2022b].”
BackgroundMethodInfluentialLearning functional properties of proteins with language models
Ünsal, S., Ataş, H., Albayrak, M., Turhan, K., Acar, A. C., & Doğan, T. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00457-9 · Cited by 206
“The disentanglement of protein representations is a new subject, and only a few representation model developers have explored this issue thus far 61,37 .”
BackgroundCui, Y., Chen, Y., Sun, J., Zhu, T., Pang, H., Li, C., Geng, W.-C., & Wu, B. (2024) · Nature Communications · doi:10.1038/s41467-024-45662-9 · Cited by 221
AbLang: an antibody language model for completing antibody sequences
Olsen, T. H., Moal, I. H., & Deane, C. M. (2022) · Bioinformatics Advances · doi:10.1093/bioadv/vbac046 · Cited by 198
“Such protein language models require vast amounts of training data and so far have tended to use all protein sequences and therefore be general protein representations (Alley et al. , 2019, Elnaggar et al ., 2021, Rives et al ., 2021).”
BackgroundControllable protein design with language models
Ferruz, N., & Höcker, B. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00499-z · Cited by 209
“Inspired by their success, Alley et al. utilized this architecture for a protein language model, UniRep, which predicted sequence stability with higher accuracy than previous methods 24 .”
MethodProteInfer, deep neural networks for protein functional inference
Sanderson, T., Bileschi, M. L., Belanger, D., & Colwell, L. J. (2023) · eLife · doi:10.7554/elife.80942 · Cited by 197
“This demonstrates the power of embeddings to simplify other studies with limited labelled data, as has been observed in recent work ( Alley et al., 2019 ; Biswas et al., 2021 ).”
A general model to predict small molecule substrates of enzymes based on machine and deep learning
Kroll, A., Ranjan, S., Engqvist, M. K. M., & Lercher, M. J. (2023) · Nature Communications · doi:10.1038/s41467-023-38347-2 · Cited by 193
“In order to create a single fi xed-length numerical representation of the whole protein, one typically calculates the element-wise mean across all amino acid representations 34,35,49 .”
BackgroundTurnover number predictions for kinetically uncharacterized enzymes using machine and deep learning
Kroll, A., Rousset, Y., Hu, X.-P., Liebrand, N. A., & Lercher, M. J. (2023) · Nature Communications · doi:10.1038/s41467-023-39840-4 · Cited by 186
Machine learning for biochemical engineering: A review
Mowbray, M., Savage, T., Wu, C., Song, Z., Cho, B. A., Del Rio-Chanona, E. A., & Zhang, D. (2021) · Biochemical Engineering Journal · doi:10.1016/j.bej.2021.108054 · Cited by 188
Prihoda, D., Maamary, J., Waight, A., Juan, V., Fayadat-Dilman, L., Svozil, D., & Bitton, D. A. (2022) · mAbs · doi:10.1080/19420862.2021.2020203 · Cited by 190
“Such representations can be leveraged for transfer learning to specific problems with limited training sets.(23) In antibody discovery, AbLSTM(26) has demonstrated the ability of deep learning to distinguish between human and nonhuman antibody sequences.”
BackgroundGenerative chemistry: drug discovery with deep learning generative models
Bian, Y., & Xie, X.-Q. (2021) · Journal of Molecular Modeling · doi:10.1007/s00894-021-04674-8 · Cited by 206
Deep learning for drug repurposing: Methods, databases, and applications
Pan, X., Lin, X., Cao, D., Zeng, X., Yu, P. S., He, L., Nussinov, R., & Cheng, F. (2022) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.1597 · Cited by 202
“Different from k-gram, a unified representation method[71] was designed to apply RNN to learn statistical representations of proteins from unlabeled amino acid sequences, which are semantically rich and structurally, evolutionarily and biophysically grounded.”
MethodMachine Learning Applied to Predicting Microorganism Growth Temperatures and Enzyme Catalytic Optima
Li, G., Rabe, K. S., Nielsen, J., & Engqvist, M. K. M. (2019) · ACS Synthetic Biology · doi:10.1021/acssynbio.9b00099 · Cited by 185
Ankh ☥: Optimized Protein Language Model Unlocks General-Purpose ModellingPreprint
Elnaggar, A., Essam, H., Salah-Eldin, W., Moustafa, W., Elkerdawy, M., Rochereau, C., & Rost, B. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.16.524265 · Cited by 243
Applications of transformer-based language models in bioinformatics: a survey
Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023) · Bioinformatics Advances · doi:10.1093/bioadv/vbad001 · Cited by 176
Language models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181
Embeddings from protein language models predict conservation and variant effects
(2021) · doi:10.1007/s00439-021-02411-y · Cited by 174
“…been transferred to protein sequences by learning to predict masked or missing amino acids using large databases of raw protein sequences as input (Alley et al. 2019; Bepler and Berger 2019a, 2021; Elnaggar et al. 2021; Heinzinger et al. 2019; Madani et al. 2020; Ofer et al. 2021; Rao et al.…”
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2023) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 238
ECNet is an evolutionary context-integrated deep learning framework for protein engineering
Luo, Y., Jiang, G., Yu, T., Liu, Y., Vo, L. T., Ding, H., Su, Y., Qian, W. W., Zhao, H., & Peng, J. (2021) · Nature Communications · doi:10.1038/s41467-021-25976-8 · Cited by 174
Rahban, M., Zolghadri, S., Salehi, N., Ahmad, F., Haertlé, T., Rezaei‐Ghaleh, N., Sawyer, L., & Saboury, A. A. (2022) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2022.06.154 · Cited by 178
Rational design of enzyme activity and enantioselectivity
Song, Z., Zhang, Q., Wu, W., Pu, Z., & Yu, H. (2023) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2023.1129149 · Cited by 183
“Although machine learning has achieved several successful cases in enzyme engineering (Yang et al., 2018; Alley et al., 2019; Yang et al., 2019; Thean et al., 2022), there is currently no universal method for a wide range of applications.”
BackgroundLearning meaningful representations of protein sequences
Detlefsen, N. S., Hauberg, S., & Boomsma, W. (2022) · Nature Communications · doi:10.1038/s41467-022-29443-w · Cited by 171
“We use data for remote homology detection from [3], fluorescence landscape prediction from [4] and for stability landscape prediction from [5], all spanning multiple protein families.”
BackgroundMethodInfluentialAdvances in machine learning for directed evolution
Wittmann, B. J., Johnston, K. E., Wu, Z., & Arnold, F. H. (2021) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2021.01.008 · Cited by 166
Sadeghi, D., Shoeibi, A., Ghassemi, N., Moridian, P., Khadem, A. A., Alizadehsani, R., Teshnehlab, M., Górriz, J. M., Khozeimeh, F., Zhang, Y., et al. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105554 · Cited by 178
Neural networks to learn protein sequence–function relationships from deep mutational scanning data
Gelman, S., Fahlberg, S. A., Heinzelman, P., Romero, P. A., & Gitter, A. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2104878118 · Cited by 164
“Other important machine learning advances relevant to protein engineering include generative modeling to sample non-natural protein sequences [23, 29, 30], language models to learn protein representations from diverse natural sequences [31–34], and strategies to incorporate machine learning predictions into directed e…”
BackgroundRahban, M., Ahmad, F., Piatyszek, M. A., Haertlé, T., Saso, L., & Saboury, A. A. (2023) · RSC Advances · doi:10.1039/d3ra06476j · Cited by 166
“162 Wang et al. 163 developed an innovative computational method named BayeStab, speci cally designed for predicting changes in protein thermostability caused by mutations.”
MethodNavigating the landscape of enzyme design: from molecular simulations to machine learning
Zhou, J., & Huang, M. (2024) · Chemical Society Reviews · doi:10.1039/d4cs00196f · Cited by 168
“286 However, it is worth noting that if the small dataset is very divergent from the large dataset, the knowledge transferred may not be relevant, leading to poor performance.”
Applications of artificial intelligence to enzyme and pathway design for metabolic engineering
Jang, W. D., Kim, G. B., Kim, Y., & Lee, S. Y. (2021) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2021.07.024 · Cited by 159
Makowski, E. K., Kinnunen, P. C., Huang, J., Wu, L., Smith, M. D., Wang, T., Desai, A. A., Streu, C. N., Zhang, Y., Zupancic, J. M., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-31457-3 · Cited by 158
Molecular de-extinction of ancient antimicrobial peptides enabled by machine learning
Maasch, J. R. M. A., Torres, M. D. T., Melo, M. C. R., & de la Fuente‐Núñez, C. (2023) · Cell Host & Microbe · doi:10.1016/j.chom.2023.07.001 · Cited by 161
Wen, B., Zeng, W., Liao, Y., Shi, Z., Savage, S. R., Jiang, W., & Zhang, B. (2020) · PROTEOMICS · doi:10.1002/pmic.201900335 · Cited by 160
Luo, S., Su, Y., Peng, J., Wang, S., Peng, J., & Ma, J. (2022) · neural information processing systems · doi:10.52202/068431-0709 · Cited by 170
Single-sequence protein structure prediction using supervised transformer protein language models
Wang, W., Peng, Z., & Yang, J. (2022) · Nature Computational Science · doi:10.1038/s43588-022-00373-3 · Cited by 150
Metadata
- Authors
- Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M.
- Affiliations
- Harvard University (US)
- Corresponding author
- Church, George M.
- Venue
- Nature Methods
- Published
- 2019-10-21
- Volume
- 16
- Issue
- 12
- Pages
- 1315-1322
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC7067682
- OpenAlex
- W2980789587
- Cited by
- 1,131
- References
- 80
- Citations by year
- 2026: 130 · 2025: 170 · 2024: 201 · 2023: 197 · 2022: 188 · 2021: 159 · 2020: 69 · 2019: 17
- FWCI
- 35.71
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Nature Methods (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- Funders
- Amazon Web Services Cloud Credits for Research
- Center for Effective Altruism
- U.S. Department of Health & Human Services | National Institutes of Health (T32HG002295, U54-CA225088)
- National Science Foundation GRFP Fellowship
- U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (P50GM107618)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- rational protein design, protein stability prediction, de novo protein design, protein mutants, recurrent neural networks, protein function prediction
- MeSH terms
- Deep Learning, Amino Acid Sequence, Mutation, Protein Engineering, Protein Stability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
