Unified rational protein engineering with sequence-based deep representation learning
Open accessPublished version of a preprint
Abstract
Rational protein engineering requires a holistic understanding of protein function. Here, we apply deep learning to unlabeled amino-acid sequences to distill the fundamental features of a protein into a statistical representation that is semantically rich and structurally, evolutionarily and biophysically grounded. We show that the simplest models built on top of this unified representation (UniRep) are broadly applicable and generalize to unseen regions of sequence space. Our data-driven approach predicts the stability of natural and de novo designed proteins, and the quantitative function of molecularly diverse mutants, competitively with the state-of-the-art methods. UniRep further enables two orders of magnitude efficiency improvement in a protein engineering task. UniRep is a versatile summary of fundamental protein features that can be applied across protein engineering informatics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Language models learn summary representations that can be used to capture sequence relationships and link sequence to function via transfer learning.
“They also learn summary representations, powerful features that can be used to better capture sequence relationships and link sequence to function via transfer learning 27,28,32-35.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [28]
Various language model-based protein embedding methods have been applied to protein prediction problems through transfer learning, including protein phenotype prediction, residue-residue contact prediction, fold recognition, protein-protein and protein-drug interaction prediction.
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [28]
Deep learning architectures originally developed for natural language processing have been applied successfully to create numerical protein representations from amino acid sequences.
“In recent years, deep learning architectures that were originally developed for natural language processing (NLP) tasks, such as translating a sentence from one language into another, have been applied successfully to the creation of numerical protein representations from amino acid sequences 17 , 29 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [29]
These fingerprints are used together with a 1900-dimensional UniRep vector as input for a gradient boosted decision tree model to predict the KM value for an enzyme-substrate pair.
“These fingerprints are used together with a 1900-dimensioanl enzyme representation, called UniRep vector 29 , as the input for a gradient boosted decision tree model 34 to predict the K M value for an enzyme-substrate pair.”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1× · see citing works
“We used basic agglomerative clustering (implemented in Python with sklearn 73 ) with the average linkage metric and Euclidean distance for all vector representations, and a precomputed Levenshtein distance matrix using the Python Levenshtein package for the sequence-only control.”
MethodBerman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196 · cited 4× · see citing works
“We computed the full sequence of hidden states at every position of 3,720 single-domain proteins from the SCOP database 60 (for which we had secondary structure annotation for each of the positions obtained from the PDB 80 ).”
MethodInfluential(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320 · see citing works
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251 · see citing works
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 5,869 · see citing works
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
Valiev, M., Bylaska, E. J., Govind, N., Kowalski, K., Straatsma, T. P., van Dam, H. J. J., Wang, D., Nieplocha, J., Aprà, E., Windus, T. L., et al. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2010.04.018 · Cited by 5,794 · see citing works
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · see citing works
Molecular dynamics simulations of biomolecules
Karplus, M., & McCammon, J. (2002) · Nature Structural Biology · doi:10.1038/nsb0902-646 · Cited by 3,401 · see citing works
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
Gómez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018) · ACS Central Science · doi:10.1021/acscentsci.7b00572 · Cited by 3,106 · see citing works
Multiple Sequence Alignment Using ClustalW and ClustalX
Thompson, J., Gibson, T. J., & Higgins, D. G. (2002) · Current Protocols in Bioinformatics · doi:10.1002/0471250953.bi0203s00 · Cited by 2,989 · cited 1× · see citing works
“Remaining sequences were then multiply aligned using ClustalW (gap open penalty = 10, gap extension penalty = 0.1) 78 .”
MethodInfluentialLinguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913 · see citing works
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · see citing works
Learning to forget: continual prediction with LSTM
Gers, F. A. (1999) · doi:10.1049/cp:19991218 · Cited by 2,436 · see citing works
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 2,409 · see citing works
Insights into the phylogeny and coding potential of microbial dark matter
Rinke, C., Schwientek, P., Sczyrba, A., Ivanova, N. N., Anderson, I. J., Cheng, J.-F., Darling, A., Malfatti, S., Swan, B. K., Gies, E. A., et al. (2013) · Nature · doi:10.1038/nature12352 · Cited by 2,394 · see citing works
UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches
Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B., Wu, C. H., & the UniProt Consortium (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu739 · Cited by 1,928 · see citing works
Recovery of nearly 8,000 metagenome-assembled genomes substantially expands the tree of life
Parks, D. H., Rinke, C., Chuvochina, M., Chaumeil, P.-A., Woodcroft, B. J., Evans, P. N., Hugenholtz, P., & Tyson, G. W. (2017) · Nature Microbiology · doi:10.1038/s41564-017-0012-7 · Cited by 1,882 · see citing works
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules.
Gómez‐Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera‐Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru‐Guzik, A. (2018) · Cambridge University Engineering Department Publications Database · doi:10.17863/cam.34134 · Cited by 1,813 · see citing works
Protein Structure Prediction Using Rosetta
Rohl, C. A., Strauss, C. E. M., Misura, K. M. S., & Baker, D. A. (2004) · Methods in enzymology on CD-ROM/Methods in enzymology · doi:10.1016/s0076-6879(04)83004-0 · Cited by 1,711 · see citing works
Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Glorot, X., Bordes, A., & Bengio, Y. (2011) · Cited by 1,569 · see citing works
Understanding inverse document frequency: on theoretical arguments for IDF
Robertson, S. E. (2004) · Journal of Documentation · doi:10.1108/00220410410560582 · Cited by 1,550 · see citing works
The Rosetta All-Atom Energy Function for Macromolecular Modeling and Design
Alford, R. F., Leaver-Fay, A., Jeliazkov, J. R., O’Meara, M. J., DiMaio, F. P., Park, H., Shapovalov, M. V., Renfrew, P. D., Mulligan, V. K., Kappel, K., et al. (2017) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.7b00125 · Cited by 1,532 · see citing works
The coming of age of de novo protein design
Huang, P.-S., Boyken, S. E., & Baker, D. (2016) · Nature · doi:10.1038/nature19946 · Cited by 1,478 · see citing works
Exploring protein fitness landscapes by directed evolution
Romero, P. A., & Arnold, F. H. (2009) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm2805 · Cited by 1,163 · see citing works
Stability of Protein Pharmaceuticals: An Update
Manning, M. C., Chou, D. K., Murphy, B. M., Payne, R. W., & Katayama, D. S. (2010) · Pharmaceutical Research · doi:10.1007/s11095-009-0045-6 · Cited by 1,103 · see citing works
Methods for the directed evolution of proteins
Packer, M. S., & Liu, D. R. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3927 · Cited by 1,002 · see citing works
Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics
Asgari, E., & Mofrad, M. R. K. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141287 · Cited by 817 · see citing works
Mutation effects predicted from sequence co-variation
Hopf, T. A., Ingraham, J. B., Poelwijk, F. J., Schärfe, C. P. I., Springer, M., Sander, C., & Marks, D. S. (2017) · Nature Biotechnology · doi:10.1038/nbt.3769 · Cited by 775 · see citing works
Fox, N. K., Brenner, S. E., & Chandonia, J.-M. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1240 · Cited by 762 · see citing works
Scalable Bayesian Optimization Using Deep Neural NetworksPreprint
Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., Patwary, M. A., Prabhat, & Adams, R. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.05700 · Cited by 702 · cited 1× · see citing works
“While the high dimensionality and discrete nature of sequence space make sequence search a challenging and unsolved problem, we are optimistic about how UniRep-powered models may combine with ongoing work in simulated annealing 3 , adaptive sampling 43 and Bayesian optimization 44–50 .”
BackgroundThe Growing and Glowing Toolbox of Fluorescent and Photoactive Proteins
Rodriguez, E. A., Campbell, R. E., Lin, J. Y., Lin, M. Z., Miyawaki, A., Palmer, A. E., Shu, X., Zhang, J., & Tsien, R. Y. (2016) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2016.09.010 · Cited by 622 · see citing works
Deep generative models of genetic variation capture the effects of mutations
Riesselman, A. J., Ingraham, J. B., & Marks, D. S. (2018) · Nature Methods · doi:10.1038/s41592-018-0138-4 · Cited by 601 · see citing works
Local fitness landscape of the green fluorescent protein
Sarkisyan, K. S., Bolotin, D. A., Meer, M. V., Usmanova, D. R., Mishin, A. S., Sharonov, G. V., Ivankov, D. N., Bozhanova, N. G., Baranov, M. S., Soylemez, O., et al. (2016) · Nature · doi:10.1038/nature17995 · Cited by 591 · see citing works
Park, H., Bradley, P. H., Greisen, P. J., Liu, Y., Mulligan, V. K., Kim, D. E., Baker, D. A., & DiMaio, F. (2016) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.6b00819 · Cited by 564 · see citing works
Improving catalytic function by ProSAR-driven enzyme evolution
Fox, R. J., Davis, S. C., Mundorff, E. C., Newman, L. M., Gavrilovic, V., Ma, S. K., Chung, L., Ching, C., Tam, S., Muley, S., et al. (2007) · Nature Biotechnology · doi:10.1038/nbt1286 · Cited by 540 · see citing works
HOMSTRAD: A database of protein structure alignments for homologous families
Mizuguchi, K., Deane, C. M., Blundell, T. L., & Overington, J. P. (1998) · Protein Science · doi:10.1002/pro.5560071126 · Cited by 501 · see citing works
Global analysis of protein folding using massively parallel design, synthesis, and testing
Rocklin, G. J., Chidyausiku, T. M., Goreshnik, I., Ford, A., Houliston, S., Lemak, A., Carter, L., Ravichandran, R., Mulligan, V. K., Chevalier, A., et al. (2017) · Science · doi:10.1126/science.aan0693 · Cited by 501 · see citing works
Bommarius, A. S., & Paye, M. F. (2013) · Chemical Society Reviews · doi:10.1039/c3cs60137d · Cited by 493 · see citing works
Zdobnov, E. M., Tegenfeldt, F., Kuznetsov, D., Waterhouse, R. M., Simão, F. A., Ioannidis, P., Seppey, M., Loetscher, A., & Kriventseva, E. V. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw1119 · Cited by 438 · see citing works
Resurrecting ancient genes: experimental analysis of extinct molecules
Thornton, J. W. (2004) · Nature Reviews Genetics · doi:10.1038/nrg1324 · Cited by 399 · see citing works
Diversity and Evolution of Coral Fluorescent Proteins
Alieva, N. O., Konzen, K. A., Field, S. F., Meleshkevitch, E. A., Hunt, M. E., Ramirez, V. B., Miller, D. J., Wiedenmann, J., Salih, A., & Matz, M. V. (2008) · PLoS ONE · doi:10.1371/journal.pone.0002680 · Cited by 378 · see citing works
End-to-End Differentiable Learning of Protein Structure
AlQuraishi, M. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.006 · Cited by 371 · see citing works
Self-citationEpistasis as the primary factor in molecular evolution
Breen, M. S., Kemena, C., Vlasov, P. K., Notredame, C., & Kondrashov, F. A. (2012) · Nature · doi:10.1038/nature11510 · Cited by 354 · see citing works
De novo design of a fluorescence-activating β-barrel
Dou, J., Vorobieva, A. A., Sheffler, W., Doyle, L. A., Park, H., Bick, M. J., Mao, B., Foight, G. W., Lee, M. Y., Gagnon, L. A., et al. (2018) · Nature · doi:10.1038/s41586-018-0509-0 · Cited by 351 · see citing works
Navigating the protein fitness landscape with Gaussian processes
Romero, P. A., Krause, A., & Arnold, F. H. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1215251110 · Cited by 346 · see citing works
Simon, J. R., Carroll, N. J., Rubinstein, M., Chilkoti, A., & López, G. P. (2017) · Nature Chemistry · doi:10.1038/nchem.2715 · Cited by 344 · see citing works
Protein tolerance to random amino acid change.
Guo, H., Choe, J., & Loeb, L. (2004) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.0403255101 · Cited by 341 · see citing works
Learning to Generate Reviews and Discovering Sentiment
Radford, A., Jozefowicz, R., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1704.01444 · Cited by 330 · see citing works
Engineering an allosteric transcription factor to respond to new ligands
Taylor, N. D., Garruss, A. S., Moretti, R., Chan, S., Arbing, M. A., Cascio, D., Rogers, J. K., Isaacs, F. J., Kosuri, S., Baker, D., et al. (2016) · Nature Methods · doi:10.1038/nmeth.3696 · Cited by 328 · see citing works
Self-citationSTRUM: structure-based prediction of protein stability changes upon single-point mutation
Quan, L., Lv, Q., & Zhang, Y. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw361 · Cited by 324 · see citing works
Learned protein embeddings for machine learning
Yang, K., Wu, Z., Bedbrook, C. N., & Arnold, F. H. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty178 · Cited by 316 · see citing works
Multiplicative LSTM for sequence modellingPreprint
Krause, B., Murray, I., Renals, S., & Lu, L. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07959 · Cited by 232 · cited 2× · see citing works
“We considered the mLSTM 61 , LSTM 62 and gated recurrent unit 63 for candidate RNN representation learners.”
BackgroundMethodLarge-scale determination of previously unsolved protein structures using evolutionary information
Ovchinnikov, S., Kinch, L., Park, H., Liao, Y., Pei, J., Kim, D. E., Kamisetty, H., Grishin, N. V., & Baker, D. (2015) · eLife · doi:10.7554/elife.09248 · Cited by 222 · see citing works
Quantitative Missense Variant Effect Prediction Using Large-Scale Mutagenesis Data
Gray, V. E., Hause, R. J., Luebeck, J., Shendure, J., & Fowler, D. M. (2017) · Cell Systems · doi:10.1016/j.cels.2017.11.003 · Cited by 205 · see citing works
OXBench: A benchmark for evaluation of protein multiple sequence alignment accuracy
Raghava, G. P. S., Searle, S. M. J., Audley, P. C., Barber, J. D., & Barton, G. J. (2003) · BMC Bioinformatics · doi:10.1186/1471-2105-4-47 · Cited by 204 · cited 3× · see citing works
“To confirm our cluster assignments, we picked a small set of three cytochromeoxidase (1,2,3) families from OXBench, eight proteins in total.”
InfluentialDirect measurement of DNA affinity landscapes on a high-throughput sequencing instrument
Nutiu, R., Friedman, R. C., Luo, S., Khrebtukova, I., Silva, D., Li, R., Zhang, L., Schroth, G. P., & Burge, C. B. (2011) · Nature Biotechnology · doi:10.1038/nbt.1882 · Cited by 194 · see citing works
NGLview–interactive molecular graphics for Jupyter notebooks
Nguyen, H., Case, D. A., & Rose, A. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx789 · Cited by 164 · see citing works
ProteinNet: a standardized data set for machine learning of protein structure
AlQuraishi, M. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2932-0 · Cited by 141 · see citing works
Self-citationMultiplexed gene synthesis in emulsions for exploring protein functional landscapes
Plesa, C., Sidore, A. M., Lubock, N. B., Zhang, D., & Kosuri, S. (2018) · Science · doi:10.1126/science.aao5167 · Cited by 124 · see citing works
Bedbrook, C. N., Yang, K., Rice, A. J., Gradinaru, V., & Arnold, F. H. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005786 · Cited by 112 · see citing works
Deep Recurrent Neural Network for Protein Function Prediction from SequencePreprint
Liu, X. L. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/103994 · Cited by 106 · see citing works
Multiplex single-molecule interaction profiling of DNA-barcoded proteins
Gu, L., Li, C., Aach, J., HILL, D. E., Vidal, M., & Church, G. M. (2014) · Nature · doi:10.1038/nature13761 · Cited by 80 · see citing works
Self-citationJuárez, J. F., Lecube-Azpeitia, B., Brown, S. L., Johnston, C. D., & Church, G. M. (2018) · Nature Communications · doi:10.1038/s41467-018-05525-6 · Cited by 66 · cited 1× · see citing works
“14), evoking deep protein design, similar to previous work with small molecules 48 and proteins 43 .”
ResultSelf-citationProtein remote homology detection based on bidirectional long short-term memory
Li, S., Chen, J., & Liu, B. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1842-2 · Cited by 65 · see citing works
Computational protein design: a review
Ivan Coluzza, B. (2017) · Journal of Physics Condensed Matter · doi:10.1088/1361-648x/aa5c76 · Cited by 53 · see citing works
Unified rational protein engineering with sequence-only deep representation learningPreprint
Alley, E. C., Khimulya, G., Biswas, S., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/589333 · Cited by 50 · see citing works
Self-citationMotif kernel generated by genetic programming improves remote homology and fold detection
Håndstad, T., Hestnes, A. J., & Sætrom, P. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-23 · Cited by 49 · see citing works
The Future of Multiplexed Eukaryotic Genome Engineering
Thompson, D. B., Aboulhouda, S., Hysolli, E., Smith, C. J., Wang, S., Castanon, O., & Church, G. M. (2017) · ACS Chemical Biology · doi:10.1021/acschembio.7b00842 · Cited by 40 · see citing works
Self-citationKwon, W. S., Silva, N. A. D., & Kellis, J. T. (1996) · Protein Engineering Design and Selection · doi:10.1093/protein/9.12.1197 · Cited by 37 · see citing works
A Model of Substitution Trajectories in Sequence Space and Long-Term Protein Evolution
Usmanova, D. R., Luca Ferretti, A., Povolotskaya, I., Vlasov, P. K., & Kondrashov, F. A. (2014) · Molecular Biology and Evolution · doi:10.1093/molbev/msu318 · Cited by 33 · see citing works
End-to-end differentiable learning of protein structurePreprint
AlQuraishi, M. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/265231 · Cited by 33 · cited 9× · see citing works
“Instead of using the predicted structures of the RGN, we extracted the internal state learned by the model for each sequence + PSSM combination (3,200 dimensions corresponding to the last outputs of two bidirectional LSTMs, each comprised 800 units per direction).”
BackgroundMethodInfluentialSelf-citationToward machine-guided design of proteinsPreprint
Biswas, S., Kuznetsov, G., Ogden, P. J., Conway, N. J., Adams, R. P., & Church, G. M. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/337154 · Cited by 29 · cited 7× · see citing works
“…representation learning for proteins via RNNs. Unlike other approaches to representing proteins, namely as one-hot-encoded matrices as in Biswas et al. 3 , RNNs produce fixed-length representations for arbitrary-length proteins by extracting the hidden state passed forward along a sequence.”
BackgroundResultInfluentialSelf-citationDeep Semantic Protein Representation for Annotation, Discovery, and EngineeringPreprint
Schwartz, A. S., Hannum, G., Dwiel, Z., Smoot, M. E., Grant, A. R., Knight, J. M., Becker, S. A., Eads, J. R., LaFave, M. C., Eavani, H., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/365965 · Cited by 24 · see citing works
Experimental assay of a fitness landscape on a macroevolutionary scalePreprint
Pokusaeva, V. O., Usmanova, D., Putintseva, E., Espinar, L., Sarkisyan, K., Mishin, A., Bogatyreva, N. S., Ivankov, D., Akopyan, A., Avvakumov, S. Y., et al. (2017) · bioRxiv · doi:10.1101/222778 · Cited by 12 · see citing works
Soft Ngram Representation and Modeling for Protein Remote Homology Detection
Lovato, P., Cristani, M., & Bicego, M. (2016) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2016.2595575 · Cited by 9 · see citing works
Doan, A., Halevy, A. Y., & Ives, Z. G. (2012) · Elsevier eBooks · doi:10.1016/b978-0-12-416044-6.00004-1 · Cited by 8 · see citing works
Tree Alignment Based on Needleman-Wunsch Algorithm for Sensor Selection in Smart Homes
Chua, S.-L., & Foo, L. K. (2017) · Sensors · doi:10.3390/s17081902 · Cited by 6 · see citing works
De novo design of a fluorescence-activating beta barrel - BB1
Dou, J., Vorobieva, A. A., Sheffler, W., Doyle, L. A., Park, H., Bick, M. J., Mao, B., Foight, G., Lee, M., Carter, L., et al. (2018) · Worldwide Protein Data Bank · doi:10.2210/pdb6d0t/pdb · Cited by 5 · see citing works
tlambert03/FPbase: Initial Release
Lambert, T., & Bot, P. I. (2018) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1244328 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- [27]Doan A, Halevy A & Ives Z in Principles of Data Integration 95– 119 (Elsevier, 2012).
- [43]Brookes DH, Park H & Listgarten J Conditioning by adaptive sampling for robust design. Proc. Machine Learn. Res 97, 773– 782 (2019).
- [45]Hernández-Lobato JM, Requeima J, Pyzer-Knapp EO & Aspuru-Guzik A Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. Preprint at arXiv https://arxiv.org/abs/1706.01825 (2017). · arXiv
- [46]Snoek J, Larochelle H & Adams RP in Advances in Neural Information Processing Systems Vol. 25 (eds. Pereira F) 2951– 2959 (Curran Associates, Inc., 2012).
- [47]Griffiths R-R & Hernández-Lobato JM Constrained Bayesian optimization for automaticchemical design. Preprint at arXiv https://arxiv.org/abs/1709.05501 (2017). · arXiv
- [49]Yang KK, Chen Y, Lee A & Yue Y Batched stochastic Bayesian optimization via combinatorial constraints design. Preprint at arXiv https://arxiv.org/abs/1904.08102 (2019). · arXiv
- [50]González J, Longworth J, James DC & Lawrence ND Bayesian optimization for synthetic gene design. Preprint at arXiv https://arxiv.org/abs/1505.01627 (2015). · arXiv
- [54]Jouppi NP In-datacenter performance analysis of a tensorprocessing unit. In Proc. 44th Annual International Symposium of Computer Architecture Vol. 45, 1– 12 (ACM, 2017).
- [64]Salimans T & Kingma DP Weight normalization: a simple reparameterization to accelerate training of deep neural networks. Preprint at arXiv https://arxiv.org/abs/1602.07868 (2016). · arXiv
Cited by
Sorted by citations
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212 · see citing works
“Alley et al. also made strides in the area of protein analysis, creating statistical summaries that capture key properties of proteins and help neural networks learn with less data 43 .”
BackgroundSignalP 6.0 predicts all five types of signal peptides using protein language models
Teufel, F., Armenteros, J. J. A., Johansen, A. R., Gíslason, M. H., Pihl, S. I., Tsirigos, K. D., Winther, O., Brunak, S., von HEIJNE, G., & Nielsen, H. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01156-3 · Cited by 2,999 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · see citing works
A guide to machine learning for biologists
Greener, J. G., Kandathil, S. M., Moffat, L., & Jones, D. T. (2022) · Nature Reviews Molecular Cell Biology · doi:10.1038/s41580-021-00407-0 · Cited by 2,127 · see citing works
“biological sequences Protein engineering [68] Variable input size Long training times”
BackgroundSignalP 6.0 predicts all five types of signal peptides using protein language models
Teufel, F., Armenteros, J. J. A., Johansen, A. R., Gíslason, M. H., Pihl, S. I., Tsirigos, K. D., Winther, O., Brunak, S., von HEIJNE, G., & Nielsen, H. (2022) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000524414 · Cited by 1,905 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
“, 2020), biology (Alley et al., 2019; Rives et al., 2021), and even across multiple modalities (Das et al.”
BackgroundLarge language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01618-2 · Cited by 1,078 · see citing works
Machine-learning-guided directed evolution for protein engineering
Yang, K., Wu, Z., & Arnold, F. H. (2019) · Nature Methods · doi:10.1038/s41592-019-0496-6 · Cited by 1,082 · see citing works
ProteinBERT: a universal deep-learning model of protein sequence and function
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., & Linial, M. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac020 · Cited by 1,009 · see citing works
ProtGPT2 is a deep unsupervised language model for protein design
Ferruz, N., Schmidt, S., & Höcker, B. (2022) · Nature Communications · doi:10.1038/s41467-022-32007-7 · Cited by 795 · see citing works
“Given both the growth of proteinsequence databases and the lack of annotation for a signi fi cant part of the protein space, protein sequences have become great candidates for unsupervised training 4,10,11 and now offer the opportunity to encode and generate protein sequences.”
BackgroundMethodSimulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937 · see citing works
Directed Evolution: Methodologies and Applications
Wang, Y., Xue, P., Cao, M., Yu, T., Lane, S., & Zhao, H. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.1c00260 · Cited by 719 · see citing works
Evaluating Protein Transfer Learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., & Song, Y. S. (2019) · doi:10.1101/676825 · Cited by 719 · see citing works
Language models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.07.09.450648 · Cited by 635 · see citing works
Deep learning for molecular design—a review of the state of the art
Elton, D. C., Boukouvalas, Z., Fuge, M. D., & Chung, P. W. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c9me00039a · Cited by 683 · see citing works
Chen, L., Tan, X., Wang, D., Zhong, F., Liu, X., Yang, T., Luo, X., Chen, K., Jiang, H., & Zheng, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa524 · Cited by 607 · see citing works
Learning the protein language: Evolution, structure, and function
Bepler, T., & Berger, B. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.017 · Cited by 483 · see citing works
Efficient evolution of human antibodies from general protein language models
Hie, B. L., Shanker, V. R., Xu, D., Bruun, T. U. J., Weidenbacher, P., Tang, S.-G., Wu, W., Pak, J., & Kim, P. S. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01763-2 · Cited by 452 · see citing works
Single-sequence protein structure prediction using a language model and deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Floristean, C., Kharkar, A., Roy, K., Rochereau, C., Ahdritz, G., Zhang, J., Church, G. M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01432-w · Cited by 455 · see citing works
Self-citationSingle-sequence protein structure prediction using language models from deep learning
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., & AlQuraishi, M. (2021) · doi:10.1101/2021.08.02.454840 · Cited by 455 · see citing works
Self-citationAhdritz, G., Bouatta, N., Floristean, C., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T. J., Berenberg, D., Fisk, I., Zanichelli, N., et al. (2024) · Nature Methods · doi:10.1038/s41592-024-02272-z · Cited by 439 · see citing works
“These include the ESMFold method for protein structure prediction 50 , which replaces MSAs with protein language models 51–53 , and FastFold, a community effort that has implemented substantial improvements including fast model-parallel training and inference 41 .”
Self-citationLearning inverse folding from millions of predicted structuresPreprint
Hsu, C., Verkuil, R., Liu, J. J., Lin, Z., Hie, B., Sercu, T., Lerer, A. K., & Rives, A. W. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.10.487779 · Cited by 434 · see citing works
Genome-wide prediction of disease variant effects with a deep protein language model
Brandes, N., Goldman, G., Wang, C. H., Ye, C., & Ntranos, V. (2023) · Nature Genetics · doi:10.1038/s41588-023-01465-0 · Cited by 459 · see citing works
Low-N protein engineering with data-efficient deep learning
Biswas, S., Khimulya, G., Alley, E. C., Esvelt, K. M., & Church, G. M. (2021) · Nature Methods · doi:10.1038/s41592-021-01100-y · Cited by 422 · see citing works
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · see citing works
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised LearningPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.12.199554 · Cited by 409 · see citing works
Expanding functional protein sequence spaces using generative adversarial networks
Repecka, D., Jauniskis, V., Karpus, L., Rembeza, E., Rokaitis, I., Zrimec, J., Poviloniene, S., Laurynenas, A., Viknander, S., Abuajwa, W., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00310-5 · Cited by 388 · see citing works
High-resolution <i>de novo</i> structure prediction from primary sequence
Wu, R., Ding, F., Wang, R., Shen, R., Zhang, X., Luo, S., Su, C., Wu, Z., Xie, Q., Berger, B., et al. (2022) · doi:10.1101/2022.07.21.500999 · Cited by 392 · see citing works
“Similar to the ESM-1b model ( 12 ), the overall architecture of OmegaPLM is a self-attention model ( 24 ) in which each token is a single amino acid.”
MethodYi, D., Bayer, T., Badenhorst, C. P. S., Wu, S., Doerr, M., Höhne, M., & Bornscheuer, U. T. (2021) · Chemical Society Reviews · doi:10.1039/d0cs01575j · Cited by 387 · see citing works
Mega-scale experimental analysis of protein folding stability in biology and design
(2023) · doi:10.1038/s41586-023-06328-6 · Cited by 350 · see citing works
“Large-scale proteolysis data have already had a key role in the development of machine learning methods for protein design and protein biophysics 19,20 .”
BackgroundRao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350 · see citing works
Learning the language of viral evolution and escape
Hie, B., Zhong, E. D., Berger, B., & Bryson, B. (2021) · Science · doi:10.1126/science.abd7331 · Cited by 361 · see citing works
De novo protein design—From new structures to programmable functions
Kortemme, T. (2024) · Cell · doi:10.1016/j.cell.2023.12.028 · Cited by 361 · see citing works
“In principle, function is also encoded in these structures and sequences, and this encoding has been used by machine learning models to generate functional proteins 21,23–25 .”
Enzyme discovery and engineering for sustainable plastic recycling
Zhu, B., Wang, D., & Wei, N. (2022) · Trends in Biotechnology · doi:10.1016/j.tibtech.2021.02.008 · Cited by 366 · see citing works
BERTology Meets Biology: Interpreting Attention in Protein Language Models
Vig, J., Madani, A., Varshney, L. R., Xiong, C., Socher, R., & Rajani, N. F. (2020) · doi:10.1101/2020.06.26.174417 · Cited by 338 · see citing works
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 345 · see citing works
“lutional networks pretrained on 3D-structure contact prediction [13,116], LSTM language models [3], as well as non-deep-”
BackgroundInfluentialTransformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341 · see citing works
“Lastly, Alley et al. (2019) presented a method of fine-tuning where a pretrained language model is further trained on the MSA of the sequence of interest (‘evotuning’).”
BackgroundMethodInfluentialUsing deep learning to annotate the protein universe
Bileschi, M. L., Belanger, D., Bryant, D. H., Sanderson, T., Carter, B., Sculley, D., Bateman, A., DePristo, M. A., & Colwell, L. J. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01179-w · Cited by 332 · see citing works
Protein design and variant prediction using autoregressive generative models
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., & Marks, D. S. (2021) · Nature Communications · doi:10.1038/s41467-021-22732-w · Cited by 320 · see citing works
Machine learning for metabolic engineering: A review
Lawson, C. E., Martí, J. M., Radivojević, T., Jonnalagadda, S. V. R., Gentz, R., Hillson, N. J., Peisert, S., Kim, J., Simmons, B. A., Petzold, C. J., et al. (2020) · Metabolic Engineering · doi:10.1016/j.ymben.2020.10.005 · Cited by 321 · see citing works
AlQuraishi, M. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz422 · Cited by 315 · see citing works
Self-citationDeep diversification of an AAV capsid protein by machine learning
Bryant, D., Bashir, A., Sinai, S., Jain, N. K., Ogden, P. J., Riley, P. F., Church, G. M., Colwell, L. J., & Kelsic, E. D. (2021) · Nature Biotechnology · doi:10.1038/s41587-020-00793-4 · Cited by 302 · see citing works
Self-citationMachine learning for antimicrobial peptide identification and design
Wan, F.-P., Wong, F., Collins, J. J., & de la Fuente-Núñez, C. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00152-x · Cited by 278 · see citing works
“Specifically, amino acid and protein features learned from protein neural language models 39,76,77 trained on large-scale protein sequences have shown strong predictive power on tasks such as predicting protein structure, stability and function after introducing mutations.”
BackgroundChakraborty, C., Bhattacharya, M., Pal, S., & Lee, S. (2023) · Current Research in Biotechnology · doi:10.1016/j.crbiot.2023.100164 · Cited by 295 · see citing works
Machine learning for functional protein design
Notin, P., Rollins, N. J., Gal, Y., Sander, C., & Marks, D. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 278 · see citing works
“This has led to a wide diversity of models, inspired by learning paradigms initially introduced in the natural language-processing literature, such as autoregressive modeling 20,52,61,141 , masked-language modeling 15,146,175,176 or seq2seq architectures 177,178 .”
Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering
Yang, J., Li, F.-Z., & Arnold, F. H. (2024) · ACS Central Science · doi:10.1021/acscentsci.3c01275 · Cited by 257 · see citing works
Elnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 305 · see citing works
Machine learning in protein structure prediction
AlQuraishi, M. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.04.005 · Cited by 252 · see citing works
Self-citationSaProt: Protein Language Modeling with Structure-aware VocabularyPreprint
Su, J., Han, C., Zhou, Y., Shan, J., Zhou, X., & Yuan, F. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.01.560349 · Cited by 251 · see citing works
“Moreover, prominent PLMs like UniRep (Al-ley et al., 2019), ProtTrans (Elnaggar et al., 2021), ESM (Rives et al., 2019; Meier et al., 2021; Rao et al., 2021; Lin et al., 2022), and Evoformer (Hu et al., 2022; Jumper et al., 2021) have showcased outstanding performance across a diverse array of tasks pertaining to prot…”
BackgroundPredictProtein - Predicting Protein Structure and Function for 29 Years
Bernhofer, M., Dallago, C., Karl, T., Satagopam, V., Heinzinger, M., Littmann, M., Olenyi, T., Qiu, J., Schütze, K., Yachdav, G., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab354 · Cited by 239 · see citing works
Recent advances in de novo protein design: Principles, methods, and applications
Pan, X., & Kortemme, T. (2021) · Journal of Biological Chemistry · doi:10.1016/j.jbc.2021.100558 · Cited by 221 · see citing works
“Moreover, the most recent developments of deep learning for protein structure prediction (41–43) foreshadow new methods in the design, taking advantage of learned principles of the protein structure (44, 45).”
BackgroundMethodAccelerating antibiotic discovery through artificial intelligence
Melo, M. C. R., Maasch, J. R. M. A., & de la Fuente-Nunez, C. (2021) · Communications Biology · doi:10.1038/s42003-021-02586-0 · Cited by 234 · see citing works
“The latter led to an embedded representation that could be used to derive a protein ’ s secondary structure, thermal stability, deep mutational scanning classi fi cation, and even the functional impact of mutations 31 .”
BackgroundMethodDeep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227 · see citing works
ProGen: Language Modeling for Protein GenerationPreprint
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi, R. R., Huang, P., & Socher, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.07.982272 · Cited by 282 · see citing works
Fine-tuning protein language models boosts predictions across diverse tasks
Schmirler, R., Heinzinger, M., & Rost, B. (2024) · Nature Communications · doi:10.1038/s41467-024-51844-2 · Cited by 229 · see citing works
Fast and Flexible Protein Design Using Deep Graph Neural Networks
Strokach, A. V., Becerra, D., Corbi‐Verge, C., Perez‐Riba, A., & Kim, P. M. (2020) · Cell Systems · doi:10.1016/j.cels.2020.08.016 · Cited by 218 · see citing works
Rapid in silico directed evolution by a protein language model with EVOLVEpro.
Jiang, K.-Y., Yan, Z.-Q., Bernardo, M. D., Sgrizzi, S. R., Villiger, L., Kayabolen, A., Kim, B. J., Carscadden, J. K., Hiraizumi, M., Nishimasu, H., et al. (2024) · Science · doi:10.1126/science.adr6006 · Cited by 248 · see citing works
Antibody structure prediction using interpretable deep learning
Ruffolo, J. A., Sulam, J., & Gray, J. J. (2022) · Patterns · doi:10.1016/j.patter.2021.100406 · Cited by 216 · see citing works
“In the recent UniRep method, RNNs were demonstrated to learn rich feature representations from protein sequences when trained on nextamino-acid prediction.(25) For our purposes, we developed an RNN encoder-decoder model(26); the encoder is a bidirectional long short-term memory (biLSTM) and the decoder is a long short…”
MethodInformed training set design enables efficient machine learning-assisted directed protein evolution
Wittmann, B. J., Yue, Y., & Arnold, F. H. (2021) · Cell Systems · doi:10.1016/j.cels.2021.07.008 · Cited by 208 · see citing works
Generating functional protein variants with variational autoencoders
Hawkins‐Hooker, A., Depardieu, F., Baur, S., Couairon, G., Chen, A., & Bikard, D. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008736 · Cited by 211 · see citing works
Machine Learning-Guided Protein Engineering
Kouba, P., Kohout, P., Haddadi, F., Bushuiev, A., Samusevich, R., Sedlář, J., Damborský, J., Pluskal, T., Šivic, J., & Mazurenko, S. (2023) · ACS Catalysis · doi:10.1021/acscatal.3c02743 · Cited by 214 · see citing works
Metabolic Engineering: Methodologies and Applications
Volk, M., Tran, V. G., Tan, S., Mishra, S., Fatma, Z., Boob, A. G., Li, H., Xue, P., Martin, T. A., & Zhao, H. (2022) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00403 · Cited by 211 · see citing works
Learning protein fitness models from evolutionary and assay-labeled data
Hsu, C., Nisonoff, H. M., Fannjiang, C., & Listgarten, J. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01146-5 · Cited by 222 · see citing works
“We assessed a total of 13 published machine learning methods for protein fitness (1) weak-positive only learning methods that rely on probability density modeling of evolutionarily related sequences (profile HMM 27 , EVmutation 23 and DeepSequence 24 ); (2) semi-supervised learning methods that rely on unlabeled UniRe…”
BackgroundMethodInfluentialCan we predict T cell specificity with digital biology and machine learning?
Hudson, D., Fernandes, R. A., Basham, M., Ogg, G. S., & Koohy, H. (2023) · Nature reviews. Immunology · doi:10.1038/s41577-023-00835-3 · Cited by 197 · see citing works
(2023) · doi:10.1101/2023.12.07.570727 · Cited by 207 · see citing works
“Hybrid models combine the respective strengths of family-specific alignment-based and family-agnostic language models, such as the MSA Transformer [Rao et al., 2021], evotuned UniRep [Alley et al., 2019a], Tranception [Notin et al., 2022a] and TranceptEVE [Notin et al., 2022b].”
BackgroundMethodInfluentialLearning functional properties of proteins with language models
Ünsal, S., Ataş, H., Albayrak, M., Turhan, K., Acar, A. C., & Doğan, T. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00457-9 · Cited by 196 · see citing works
“The disentanglement of protein representations is a new subject, and only a few representation model developers have explored this issue thus far 61,37 .”
BackgroundCui, Y., Chen, Y., Sun, J., Zhu, T., Pang, H., Li, C., Geng, W.-C., & Wu, B. (2024) · Nature Communications · doi:10.1038/s41467-024-45662-9 · Cited by 214 · see citing works
AbLang: an antibody language model for completing antibody sequences
Olsen, T. H., Moal, I. H., & Deane, C. M. (2022) · Bioinformatics Advances · doi:10.1093/bioadv/vbac046 · Cited by 198 · see citing works
“Such protein language models require vast amounts of training data and so far have tended to use all protein sequences and therefore be general protein representations (Alley et al. , 2019, Elnaggar et al ., 2021, Rives et al ., 2021).”
BackgroundControllable protein design with language models
Ferruz, N., & Höcker, B. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00499-z · Cited by 196 · see citing works
“Inspired by their success, Alley et al. utilized this architecture for a protein language model, UniRep, which predicted sequence stability with higher accuracy than previous methods 24 .”
MethodProteInfer, deep neural networks for protein functional inference
Sanderson, T., Bileschi, M. L., Belanger, D., & Colwell, L. J. (2023) · eLife · doi:10.7554/elife.80942 · Cited by 192 · see citing works
“This demonstrates the power of embeddings to simplify other studies with limited labelled data, as has been observed in recent work ( Alley et al., 2019 ; Biswas et al., 2021 ).”
A general model to predict small molecule substrates of enzymes based on machine and deep learning
Kroll, A., Ranjan, S., Engqvist, M. K. M., & Lercher, M. J. (2023) · Nature Communications · doi:10.1038/s41467-023-38347-2 · Cited by 193 · see citing works
“In order to create a single fi xed-length numerical representation of the whole protein, one typically calculates the element-wise mean across all amino acid representations 34,35,49 .”
BackgroundTurnover number predictions for kinetically uncharacterized enzymes using machine and deep learning
Kroll, A., Rousset, Y., Hu, X.-P., Liebrand, N. A., & Lercher, M. J. (2023) · Nature Communications · doi:10.1038/s41467-023-39840-4 · Cited by 186 · see citing works
Machine learning for biochemical engineering: A review
Mowbray, M., Savage, T., Wu, C., Song, Z., Cho, B. A., Del Rio-Chanona, E. A., & Zhang, D. (2021) · Biochemical Engineering Journal · doi:10.1016/j.bej.2021.108054 · Cited by 188 · see citing works
Prihoda, D., Maamary, J., Waight, A., Juan, V., Fayadat-Dilman, L., Svozil, D., & Bitton, D. A. (2022) · mAbs · doi:10.1080/19420862.2021.2020203 · Cited by 190 · see citing works
“Such representations can be leveraged for transfer learning to specific problems with limited training sets.(23) In antibody discovery, AbLSTM(26) has demonstrated the ability of deep learning to distinguish between human and nonhuman antibody sequences.”
BackgroundGenerative chemistry: drug discovery with deep learning generative models
Bian, Y., & Xie, X.-Q. (2021) · Journal of Molecular Modeling · doi:10.1007/s00894-021-04674-8 · Cited by 193 · see citing works
Deep learning for drug repurposing: Methods, databases, and applications
Pan, X., Lin, X., Cao, D., Zeng, X., Yu, P. S., He, L., Nussinov, R., & Cheng, F. (2022) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.1597 · Cited by 189 · see citing works
“Different from k-gram, a unified representation method[71] was designed to apply RNN to learn statistical representations of proteins from unlabeled amino acid sequences, which are semantically rich and structurally, evolutionarily and biophysically grounded.”
MethodMachine Learning Applied to Predicting Microorganism Growth Temperatures and Enzyme Catalytic Optima
Li, G., Rabe, K. S., Nielsen, J., & Engqvist, M. K. M. (2019) · ACS Synthetic Biology · doi:10.1021/acssynbio.9b00099 · Cited by 178 · see citing works
Ankh ☥: Optimized Protein Language Model Unlocks General-Purpose ModellingPreprint
Elnaggar, A., Essam, H., Salah-Eldin, W., Moustafa, W., Elkerdawy, M., Rochereau, C., & Rost, B. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.16.524265 · Cited by 201 · see citing works
Applications of transformer-based language models in bioinformatics: a survey
Zhang, S., Fan, R., Liu, Y., Chen, S., Liu, Q., & Zeng, W. (2023) · Bioinformatics Advances · doi:10.1093/bioadv/vbad001 · Cited by 176 · see citing works
Language models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181 · see citing works
Embeddings from protein language models predict conservation and variant effects
(2021) · doi:10.1007/s00439-021-02411-y · Cited by 174 · see citing works
“…been transferred to protein sequences by learning to predict masked or missing amino acids using large databases of raw protein sequences as input (Alley et al. 2019; Bepler and Berger 2019a, 2021; Elnaggar et al. 2021; Heinzinger et al. 2019; Madani et al. 2020; Ofer et al. 2021; Rao et al.…”
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2023) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 206 · see citing works
ECNet is an evolutionary context-integrated deep learning framework for protein engineering
Luo, Y., Jiang, G., Yu, T., Liu, Y., Vo, L. T., Ding, H., Su, Y., Qian, W. W., Zhao, H., & Peng, J. (2021) · Nature Communications · doi:10.1038/s41467-021-25976-8 · Cited by 171 · see citing works
Rahban, M., Zolghadri, S., Salehi, N., Ahmad, F., Haertlé, T., Rezaei‐Ghaleh, N., Sawyer, L., & Saboury, A. A. (2022) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2022.06.154 · Cited by 172 · see citing works
Rational design of enzyme activity and enantioselectivity
Song, Z., Zhang, Q., Wu, W., Pu, Z., & Yu, H. (2023) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2023.1129149 · Cited by 171 · see citing works
“Although machine learning has achieved several successful cases in enzyme engineering (Yang et al., 2018; Alley et al., 2019; Yang et al., 2019; Thean et al., 2022), there is currently no universal method for a wide range of applications.”
BackgroundLearning meaningful representations of protein sequences
Detlefsen, N. S., Hauberg, S., & Boomsma, W. (2022) · Nature Communications · doi:10.1038/s41467-022-29443-w · Cited by 163 · see citing works
“We use data for remote homology detection from [3], fluorescence landscape prediction from [4] and for stability landscape prediction from [5], all spanning multiple protein families.”
BackgroundMethodInfluentialAdvances in machine learning for directed evolution
Wittmann, B. J., Johnston, K. E., Wu, Z., & Arnold, F. H. (2021) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2021.01.008 · Cited by 165 · see citing works
Sadeghi, D., Shoeibi, A., Ghassemi, N., Moridian, P., Khadem, A. A., Alizadehsani, R., Teshnehlab, M., Górriz, J. M., Khozeimeh, F., Zhang, Y., et al. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105554 · Cited by 171 · see citing works
Neural networks to learn protein sequence–function relationships from deep mutational scanning data
Gelman, S., Fahlberg, S. A., Heinzelman, P., Romero, P. A., & Gitter, A. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2104878118 · Cited by 160 · see citing works
“Other important machine learning advances relevant to protein engineering include generative modeling to sample non-natural protein sequences [23, 29, 30], language models to learn protein representations from diverse natural sequences [31–34], and strategies to incorporate machine learning predictions into directed e…”
BackgroundRahban, M., Ahmad, F., Piatyszek, M. A., Haertlé, T., Saso, L., & Saboury, A. A. (2023) · RSC Advances · doi:10.1039/d3ra06476j · Cited by 166 · see citing works
“162 Wang et al. 163 developed an innovative computational method named BayeStab, speci cally designed for predicting changes in protein thermostability caused by mutations.”
MethodNavigating the landscape of enzyme design: from molecular simulations to machine learning.
Zhou, J., & Huang, M.-L. (2024) · Chemical Society Reviews · doi:10.1039/d4cs00196f · Cited by 106 · see citing works
“286 However, it is worth noting that if the small dataset is very divergent from the large dataset, the knowledge transferred may not be relevant, leading to poor performance.”
Applications of artificial intelligence to enzyme and pathway design for metabolic engineering
Jang, W. D., Kim, G. B., Kim, Y., & Lee, S. Y. (2021) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2021.07.024 · Cited by 154 · see citing works
Makowski, E. K., Kinnunen, P. C., Huang, J., Wu, L., Smith, M. D., Wang, T., Desai, A. A., Streu, C. N., Zhang, Y., Zupancic, J. M., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-31457-3 · Cited by 155 · see citing works
Molecular de-extinction of ancient antimicrobial peptides enabled by machine learning
Maasch, J. R. M. A., Torres, M. D. T., Melo, M. C. R., & de la Fuente‐Núñez, C. (2023) · Cell Host & Microbe · doi:10.1016/j.chom.2023.07.001 · Cited by 156 · see citing works
Wen, B., Zeng, W., Liao, Y., Shi, Z., Savage, S. R., Jiang, W., & Zhang, B. (2020) · PROTEOMICS · doi:10.1002/pmic.201900335 · Cited by 152 · see citing works
Luo, S., Su, Y., Peng, J., Wang, S., Peng, J., & Ma, J. (2022) · neural information processing systems · doi:10.52202/068431-0709 · Cited by 149 · see citing works
Single-sequence protein structure prediction using supervised transformer protein language models
Wang, W., Peng, Z., & Yang, J. (2022) · Nature Computational Science · doi:10.1038/s43588-022-00373-3 · Cited by 150 · see citing works
- Authors
- Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M.
- Affiliations
- Harvard University (US)
- Corresponding author
- Church, George M.
- Venue
- Nature Methods
- Published
- 2019-10-21
- Volume
- 16
- Issue
- 12
- Pages
- 1315-1322
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC7067682
- OpenAlex
- W2980789587
- Cited by
- 1,131
- References
- 80
- Citations by year
- 2026: 130 · 2025: 170 · 2024: 201 · 2023: 197 · 2022: 188 · 2021: 159 · 2020: 69 · 2019: 17
- FWCI
- 35.71
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Nature Methods (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- Funders
- Amazon Web Services Cloud Credits for Research
- Center for Effective Altruism
- U.S. Department of Health & Human Services | National Institutes of Health (T32HG002295, U54-CA225088)
- National Science Foundation GRFP Fellowship
- U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (P50GM107618)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- rational protein design, protein stability prediction, de novo protein design, protein mutants, recurrent neural networks, protein function prediction
- MeSH terms
- Deep Learning, Amino Acid Sequence, Mutation, Protein Engineering, Protein Stability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
