The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Unified rational protein engineering with sequence-based deep representation learning
Open accessPublished version of a preprint
Abstract
Rational protein engineering requires a holistic understanding of protein function. Here, we apply deep learning to unlabeled amino-acid sequences to distill the fundamental features of a protein into a statistical representation that is semantically rich and structurally, evolutionarily and biophysically grounded. We show that the simplest models built on top of this unified representation (UniRep) are broadly applicable and generalize to unseen regions of sequence space. Our data-driven approach predicts the stability of natural and de novo designed proteins, and the quantitative function of molecularly diverse mutants, competitively with the state-of-the-art methods. UniRep further enables two orders of magnitude efficiency improvement in a protein engineering task. UniRep is a versatile summary of fundamental protein features that can be applied across protein engineering informatics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“They also learn summary representations, powerful features that can be used to better capture sequence relationships and link sequence to function via transfer learning 27,28,32-35.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [28]
In the citing paper
“Since then, various language model-based protein embedding methods have been applied to these and other protein prediction problems through transfer learning, including protein phenotype prediction 28,32-34, residue-residue contact prediction 32,70, fold recognition 33, protein-protein 71,72 and protein-drug interaction prediction 35,73.”
From Learning the protein language: Evolution, structure, and function (2021) · cites this paper as [28]
In the citing paper
“In recent years, deep learning architectures that were originally developed for natural language processing (NLP) tasks, such as translating a sentence from one language into another, have been applied successfully to the creation of numerical protein representations from amino acid sequences 17 , 29 .”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [29]
In the citing paper
“These fingerprints are used together with a 1900-dimensioanl enzyme representation, called UniRep vector 29 , as the input for a gradient boosted decision tree model 34 to predict the K M value for an enzyme-substrate pair.”
From Turnover number predictions for kinetically uncharacterized enzymes using machine and deep learning (2023) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We used basic agglomerative clustering (implemented in Python with sklearn 73 ) with the average linkage metric and Euclidean distance for all vector representations, and a precomputed Levenshtein distance matrix using the Python Levenshtein package for the sequence-only control.”
MethodBerman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196 · cited 4×
“We computed the full sequence of hidden states at every position of 3,720 single-domain proteins from the SCOP database 60 (for which we had secondary structure annotation for each of the positions obtained from the PDB 80 ).”
MethodInfluential(2008) · Journal of Machine Learning Research · Cited by 33,051
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 6,122
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
Valiev, M., Bylaska, E. J., Govind, N., Kowalski, K., Straatsma, T. P., van Dam, H. J. J., Wang, D., Nieplocha, J., Aprà, E., Windus, T. L., et al. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2010.04.018 · Cited by 5,849
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 3,827
Molecular dynamics simulations of biomolecules
Karplus, M., & McCammon, J. (2002) · Nature Structural Biology · doi:10.1038/nsb0902-646 · Cited by 3,401
Research works printed but not linked: 8
- Brookes, D. H., Park, H. & Listgarten, J. Conditioning by adaptive sampling for robust design. Proc. Machine Learn. Res. 97, 773–782 (2019).
- Hernández-Lobato, J. M., Requeima, J., Pyzer-Knapp, E. O. & Aspuru-Guzik, A. Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. Preprint at arXiv https://arxiv.org/abs/1706.01825 (2017). · arXiv
- Snoek, J., Larochelle, H. & Adams, R. P. in Advances in Neural Information Processing Systems Vol. 25 (eds. Pereira, F. et al.) 2951–2959 (Curran Associates, Inc., 2012).
- Griffiths, R.-R. & Hernández-Lobato, J. M. Constrained Bayesian optimization for automaticchemical design. Preprint at arXiv https://arxiv.org/abs/1709.05501 (2017). · arXiv
- Yang, K. K., Chen, Y., Lee, A. & Yue, Y. Batched stochastic Bayesian optimization via combinatorial constraints design. Preprint at arXiv https://arxiv.org/abs/1904.08102 (2019). · arXiv
- González, J., Longworth, J., James, D. C. & Lawrence, N. D. Bayesian optimization for synthetic gene design. Preprint at arXiv https://arxiv.org/abs/1505.01627 (2015). · arXiv
- Jouppi, N. P. et al. In-datacenter performance analysis of a tensorprocessing unit. In Proc. 44th Annual International Symposium of Computer Architecture Vol. 45, 1–12 (ACM, 2017).
- Salimans, T. & Kingma, D. P. Weight normalization: a simple reparameterization to accelerate training of deep neural networks. Preprint at arXiv https://arxiv.org/abs/1602.07868 (2016). · arXiv
Cited by
Sorted by citations
Items 401–500 of 1,003
Enzymatic synthesis of chiral alcohols using ketoreductases
Yuan, Q., Ma, L., Kong, W., Liu, J., Zhang, S., Yan, J., Bai, J., He, Y., Zhou, L., Liu, Y., et al. (2024) · Catalysis Reviews · doi:10.1080/01614940.2024.2313603 · Cited by 23
pLM4CPPs: Protein Language Model-Based Predictor for Cell Penetrating Peptides
Kumar, N., Du, Z., & Li, Y. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01338 · Cited by 22
Large-scale discovery, analysis and design of protein energy landscapes
Ferrari, Á. J. R., Dixit, S. M., Thibeault, J., Garcia, M., Houliston, S., Ludwig, R. W., Notin, P., Phoumyvong, C. M., Martell, C. M., Jung, M. D., et al. (2026) · Nature · doi:10.1038/s41586-026-10465-z · Cited by 22
Vongsouthi, V., Georgelin, R., Matthews, D. S., Saunders, J. W., Lee, B. M., Ton, J., Damry, A. M., Frkic, R. L., Spence, M. A., & Jackson, C. J. (2025) · Science Advances · doi:10.1126/sciadv.ads8318 · Cited by 22
Deep Learning in Antimicrobial Peptide Prediction
Lin, C., Xiong, S., Cui, F., Zhang, Z., Shi, H. W., & Wei, L. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c00530 · Cited by 22
Efficient Exploration of Sequence Space by Sequence-Guided Protein Engineering and Design
Clifton, B. E., Kozome, D., & Laurino, P. (2022) · Biochemistry · doi:10.1021/acs.biochem.1c00757 · Cited by 22
PRECOGx: e x ploring GPCR signaling mechanisms with deep protein representations
Matic, M., Singh, G., Carli, F., Rosa, N. D. O., Miglionico, P., Magni, L., Gutkind, J. S., Russell, R. B., Inoue, A., & Raimondi, F. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac426 · Cited by 22
“ESM was shown to outperform competing methods for protein embeddings (e.g. SeqVec (21) or Unirep (22)) and similar architectures, i.e. Evoformer, form the basis of the groundbreaking protein structure prediction algorithm AlphaFold2 (23).”
Jiang, J., Li, J., Li, J., Pei, H., Li, M., Zou, Q., & Lv, Z. (2023) · Foods · doi:10.3390/foods12071498 · Cited by 22
“Based on the aforementioned analyses, the first 177D Foods 2023, 12, 1498 12 of 17 features of UniRep were selected for constructing the iUmami-DRLF predictor based on the LGBM model for subsequent studies.”
MethodInfluentialIntegration of multiple stress signals in plants using synthetic Boolean logic gates
Anderson, C. E., Ferreira, S. S., & Antunes, M. S. (2023) · PLANT PHYSIOLOGY · doi:10.1093/plphys/kiad254 · Cited by 22
“…traits, such as tolerance to environmental stress or altered morphological development (Pleiss 2006; Osbourn et al. 2012; Cirino and Qian 2013; Del Vecchio et al. 2016; Hsiao et al. 2018; Pouvreau et al. 2018; Shih 2018; Alley et al. 2019; Yang et al. 2020; Brophy 2022; Lloyd et al. 2022).”
Large-scale predicting protein functions through heterogeneous feature fusion
Zheng, R., Huang, Z., & Deng, L. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad243 · Cited by 23
“In addition to this, sequence-based protein language models have also yielded very promising results in protein bioinformatics and even in the whole protein science [25–28].”
Deep representation learning determines drug mechanism of action from cell painting images
Wong, D. R., Logan, D. J., Hariharan, S., Stanton, R. V., Clevert, D.-A., & Kiruluta, A. J. M. (2023) · Digital Discovery · doi:10.1039/d3dd00060e · Cited by 22
Deep embeddings to comprehend and visualize microbiome protein space
Odrzywolek, K., Karwowska, Z., Majta, J., Byrski, A., Milanowska-Zabel, K., & Kosciolek, T. (2022) · Scientific Reports · doi:10.1038/s41598-022-14055-7 · Cited by 22
Zhou, K., Lei, C., Zheng, J., Huang, Y., & Zhang, Z. (2023) · Plant Methods · doi:10.1186/s13007-023-01119-6 · Cited by 26
“UniRep was based on multiplicative LSTM (mLSTM) and was trained on UniRef50 [34].”
MethodLeveraging ancestral sequence reconstruction for protein representation learning
Matthews, D. S., Spence, M. A., Mater, A. C., Nichols, J., Pulsford, S. B., Sandhu, M., Kaczmarski, J. A., Miton, C. M., Tokuriki, N., & Jackson, C. J. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00935-2 · Cited by 22
“12), which requires retraining of all parameters on family or task-specific data 3,4 or spiked data scheduling, where the fine-tuning dataset size is diluted 100-fold with unrelated protein sequences to avoid forgetting of universal protein features 5 .”
Engineering CAR‐T Therapeutics for Enhanced Solid Tumor Targeting
Zhu, D., Kim, W., Lee, H., Bao, X., & Kim, P. (2025) · Advanced Materials · doi:10.1002/adma.202414882 · Cited by 24
Generalizable and scalable protein stability prediction with rewired protein generative models
Li, Z., & Luo, Y. (2025) · Nature Communications · doi:10.1038/s41467-025-67609-4 · Cited by 24
“(5))orothersequencelikelihoodmodels(e.g.,EVmutation 67 , Evotuned UniRep 69 , DeepSequence 67 ).”
Constraints on the expansion of paralogous protein families
McClune, C. J., & Laub, M. T. (2020) · Current Biology · doi:10.1016/j.cub.2020.02.075 · Cited by 21
A Sort-Seq Approach to the Development of Single Fluorescent Protein Biosensors
Koberstein, J. N., Stewart, M. L., Mighell, T. L., Smith, C. B., & Cohen, M. S. (2021) · ACS Chemical Biology · doi:10.1021/acschembio.1c00423 · Cited by 21
prPred‐DRLF: Plant R protein predictor using deep representation learning features
Wang, Y., Xu, L., Zou, Q., & Lin, C. (2021) · PROTEOMICS · doi:10.1002/pmic.202100161 · Cited by 21
“UniRep is a deep representation learning method that adopts a 1900-hidden unit mLSTM to pretrain amino acid characteristics on UniRef50 and has been found to increase the efficiency in proteinengineeringproblems[23–25].”
MethodCurrent and Emerging Tools of Computational Biology To Improve the Detoxification of Mycotoxins
Sandlin, N., Kish, D. R., Kim, J., Zaccaria, M., & Momeni, B. (2021) · Applied and Environmental Microbiology · doi:10.1128/aem.02102-21 · Cited by 21
“The traditional statistical meth-ods for ProSAR involved partial least square regression and genetic algorithm, while more recently traditional statistical methods could be replaced with recurrent neural network architectures (112).”
MethodMachine Learning at the Interface of Polymer Science and Biology: How Far Can We Go?
Gianti, E., & Percec, S. (2022) · Biomacromolecules · doi:10.1021/acs.biomac.1c01436 · Cited by 20
Data-Driven Synthetic Cell Factories Development for Industrial Biomanufacturing
Shi, Z., Liu, P., Liao, X., Mao, Z., Zhang, J., Wang, Q., Sun, J., Ma, H., & Ma, Y. (2022) · BioDesign Research · doi:10.34133/2022/9898461 · Cited by 22
Machine Learning for Protein Engineering
Johnston, K. E., Fannjiang, C., Wittmann, B. J., Hie, B., Yang, K., & Wu, Z. (2023) · Challenges and advances in computational chemistry and physics · doi:10.1007/978-3-031-37196-7_9 · Cited by 25
Insights into the inner workings of transformer models for protein function prediction
Wenzel, M. A., Grüner, E., & Strodthoff, N. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae031 · Cited by 24
Predicting adaptive immune receptor specificities by machine learning is a data generation problem
Mason, D. M., & Reddy, S. T. (2024) · Cell Systems · doi:10.1016/j.cels.2024.11.008 · Cited by 19
PredictProtein – Predicting Protein Structure and Function for 29 YearsPreprint
Bernhofer, M., Dallago, C., Karl, T., Satagopam, V., Heinzinger, M., Littmann, M., Olenyi, T., Qiu, J., Schütze, K., Yachdav, G., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.02.23.432527 · Cited by 21
Recent advances in generative biology for biotherapeutic discovery
Mock, M. L., LANGMEAD, C. J., Grandsard, P. J., Edavettal, S. C., & Russell, A. J. (2024) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2024.01.003 · Cited by 20
Banerjee, D., Jindra, M. A., Linot, A. J., Pfleger, B. F., & Maranas, C. D. (2021) · Current Research in Biotechnology · doi:10.1016/j.crbiot.2021.12.002 · Cited by 20
Learning Strategies in Protein Directed Evolution
Cadet, X. F., Gelly, J., van Noord, A., Cadet, F., & Acevedo‐Rocha, C. G. (2022) · Methods in molecular biology · doi:10.1007/978-1-0716-2152-3_15 · Cited by 22
Long, E. M., Romay, M. C., Ramstein, G. P., Buckler, E. S., & Robbins, K. R. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2022.1041925 · Cited by 20
“UniRep is a deep learning technique which characterizes protein structure (Alley et al., 2019), which we used to produce 256unit representations of each protein and its associated mutated forms (https://github.”
MethodTan, Y., Li, M., Zhou, Z., Tan, P., Yu, H., Fan, G., & Liang, H. (2024) · Journal of Cheminformatics · doi:10.1186/s13321-024-00884-3 · Cited by 22
“CPCProt [42] leverages a contrastive predictive coding loss, whereas ProGen [15, 16], UniRep [43], ProXLNet [18], ProtGPT2 [17], and Tranception [44] are pre-trained using next amino acid prediction tasks.”
MethodNon-Identifiability and the Blessings of Misspecification in Models of Molecular Fitness
Weinstein, E. N., Amin, A., Frazer, J., & Marks, D. S. (2022) · neural information processing systems · doi:10.52202/068431-0396 · Cited by 21
The Role of Protein Engineering in Biomedical Applications of Mammalian Synthetic Biology
Bojar, D., & Fussenegger, M. (2019) · Small · doi:10.1002/smll.201903093 · Cited by 20
Using Natural Language Processing to Learn the Grammar of GlycansPreprint
Bojar, D., Camacho, D. M., & Collins, J. J. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.01.10.902114 · Cited by 20
Function-guided protein design by deep manifold samplingPreprint
Gligorijević, V., Berenberg, D., Ra, S., Watkins, A. M., Kelow, S. P., Cho, K., & Bonneau, R. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.12.22.473759 · Cited by 25
“In particular, language models (LM) using bi-directional long short-term memory (LSTM) [10] and attention-based [5] architectures and trained on protein sequences have yielded useful representations for many downstream tasks, including secondary structure and contact map predictions [5], structural comparison [10], re…”
BackgroundMethodProximal Exploration for Model-guided Protein Sequence DesignPreprint
Ren, Z., Li, J., Ding, F., Zhou, Y., Ma, J., & Peng, J. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.12.487986 · Cited by 22
“…from multiple sequence alignments to predict fitness scores (Hopf et al., 2017; Riesselman et al., 2018; Luo et al., 2021) and utilizing pre-trained language models to conduct transfer learning or zero-shot inference (Rao et al., 2019; Alley et al., 2019; Meier et al., 2021; Hsu et al., 2022).”
BackgroundMethodIs Novelty Predictable?Preprint
Fannjiang, C., & Listgarten, J. (2023) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041469 · Cited by 21
“…2016; Figliuzzi et al. 2016; Hopf et al. 2017; Riesselman et al. 2018; Laine et al. 2019; Frazer et al. 2021; Shin et al. 2021; Trinquier et al. 2021) or to natural protein sequences more broadly (Alley et al. 2019; Madani et al. 2020; Meier et al. 2021; Ferruz et al. 2022; Notin et al. 2022a, b).”
BackgroundHuang, C., Zhang, L., Tang, T., Wang, H., Jiang, Y., Ren, H., Zhang, Y., Fang, J., Zhang, W., Jia, X., et al. (2024) · JACS Au · doi:10.1021/jacsau.4c00284 · Cited by 19
Wan, F.-P., Torres, M. D. T., Guan, C.-G., & de la Fuente-Núñez, C. (2025) · Nature Protocols · doi:10.1038/s41596-025-01144-w · Cited by 19
“…the distribution of sequences; in our context, the sequences can be proteins or peptides) have been trained on large-scale protein corpora 33–36 and emerging computational and artificial intelligence (AI) approaches, thousands, or even hundreds of thousands, of antimicrobial sequences can…”
Can protein expression be ‘solved’?
Baranowski, C., Martín, H. G., Oyarzún, D. A., Spinner, A., Desai, B. J., Petzold, C. J., Nikolados, E.-M., Kraatz, S. H. W., Gaber, A., Chalkley, R. J., et al. (2025) · Trends in biotechnology · doi:10.1016/j.tibtech.2025.04.021 · Cited by 21
Jing, R., Wen, T., Liao, C., Xue, L., Liu, F., Yu, L., & Luo, J. (2021) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqab086 · Cited by 19
“We selected the bidirectional long short-term memory (BiLSTM) recurrent neural network as RNN representation learner because the recent success of the BiLSTM for the sequence-based deep representation learning in protein engineering (43).”
Zhang, T., Jia, Y., Li, H., Xu, D., Zhou, J., & Wang, G. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac335 · Cited by 19
LambdaPP : Fast and accessible protein‐specific phenotype predictions
(2022) · doi:10.1002/pro.4524 · Cited by 19
“For this task, ProtT5 embeddings (24) 109 significantly outperformed all other pLM embeddings as 110 input for the same architecture (4; 9; 25; 24; 55).”
BackgroundInfluentialThermophilic Nucleic Acid Polymerases and Their Application in Xenobiology
Wang, G., Du, Y., Ma, X., Ye, F., Qin, Y., Wang, Y., Xiang, Y., Tao, R. X., & Chen, T. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms232314969 · Cited by 20
Elucidating the functional roles of prokaryotic proteins using big data and artificial intelligence
Ardern, Z., Chakraborty, S., Lenk, F., & Kaster, A. (2023) · FEMS Microbiology Reviews · doi:10.1093/femsre/fuad003 · Cited by 22
“…GO Rifaioglu et al. (2019) Strodthoff et al. (2020) Li et. al (2017) Griffith and Holehouse (2021) Hannigan et al. (2019) Hoarfrost et al. (2020) Alley et al. (2019) Cao et al. (2017) MultiPredGO DeepGOPlus DeepAdd Balrog ProtCNN DeepHiFam SmORFinder DeepEP MaSIF ONN4MST ON4ARG DeeProtGO EXPERT…”
BackgroundSurface ID: a geometry-aware system for protein molecular surface comparison
Riahi, S., Lee, J. H., Sorenson, T., Wei, S., Jäger, S., Olfati‐Saber, R., Zhou, Y., Park, A., Wendt, M., Minoux, H., et al. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad196 · Cited by 18
“In contrast to protein language model (PLM) embeddings which operate in residue or sequence level (Alley et al. 2019; Rives et al. 2021), the Surface ID embedding encodes chemical and geometric properties directly from 3D structure in a sequenceindependent manner.”
BackgroundBayesian Optimization in Drug Discovery
Colliandre, L., & Muller, C. (2023) · Methods in molecular biology · doi:10.1007/978-1-0716-3449-3_5 · Cited by 20
A robust deep learning workflow to predict CD8 + T-cell epitopes
Lee, C. H.-J., Huh, J., Buckley, P. R., Jang, M., Pinho, M. P., Fernandes, R. A., Antanaviciute, A., Simmons, A., & Koohy, H. (2023) · Genome Medicine · doi:10.1186/s13073-023-01225-z · Cited by 21
Wee, J., Chen, J., Xia, K., & Wei, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107918 · Cited by 20
“Additionally, LSTM-based models like Bepler [31] and UniRep [32] have been developed.”
MethodHyperPCM: Robust Task-Conditioned Modeling of Drug–Target Interactions
Svensson, E., Hoedt, P.-J., Hochreiter, S., & Klambauer, G. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01417 · Cited by 21
PARROT is a flexible recurrent neural network framework for analysis of large protein datasets
Griffith, D., & Holehouse, A. S. (2021) · eLife · doi:10.7554/elife.70576 · Cited by 23
“…has been a previous focus on the application of deep learning to understand folded protein stability, PARROT is demonstrably well- suited for working with intrinsically disordered protein sequences (Alley et al., 2019; Cao et al., 2019; Hoie et al., 2021; Lindorff- Larsen and Kragelund, 2021).”
BackgroundConstructing benchmark test sets for biological sequence analysis using independent set algorithms
Petti, S., & Eddy, S. R. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009492 · Cited by 19
“This is a concern because we and others are exploring increasingly 50 complex and parameter-rich models for remote sequence homology recognition that 51 can require thousands of sequences for training [8, 9, 10, 11, 12, 13].”
BackgroundLevy, B., Xu, Z., Zhao, L., Kremling, K. A. G., Altman, R., Wong, P., & Tanner, C. (2022) · Research Square · doi:10.21203/rs.3.rs-1927200/v1 · Cited by 20
Protein Representation Learning via Knowledge Enhanced Primary Structure ModelingPreprint
Zhou, H.-Y., Fu, Y., Zhang, Z., Bian, C., & Yu, Y. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.01.26.525795 · Cited by 27
“Due to a plethora of uncharacterized protein data, self-supervised pre-training (Alley et al., 2019; Rao et al., 2019) was proposed to directly learn representation from chains of amino acids, where tremendous and significant efforts were made to improve the pre-training result by scaling up the…”
BackgroundMethodVariant Effect Prediction in the Age of Machine LearningPreprint
Bromberg, Y., Prabakaran, R., Kabir, A., & Shehu, A. (2024) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041467 · Cited by 19
Wang, Y., Cheng, L., Zhang, Y., Cao, Y., & Alghazzawi, D. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf187 · Cited by 22
“In earlier work, Kroll et al . also utilized UniRep [13] to encode amino acid sequences and combined them with substrate molecular fingerprints to perform predictions of enzyme-substrate binding affinity [14].”
Discovery and protein language model-guided design of hyperactive transposases
Ivančić, D., Agudelo, A., Lindstrom-Vautrin, J., Jaraba-Wallace, J., Gallo, M., Das, R., Ragel, A., Herrero-Vicente, J., Higueras, I., Billeci, F., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02816-4 · Cited by 22
“12).”
Reconstruction of evolving gene variants and fitness from short sequencing reads
Shen, M. W., Zhao, K. T., & Liu, D. R. (2021) · Nature Chemical Biology · doi:10.1038/s41589-021-00876-6 · Cited by 18
Guidelines for designing peptoid structures: Insights from the Peptoid Data Bank
Eastwood, J. R. B., Weisberg, E. I., Katz, D., Zuckermann, R. N., & Kirshenbaum, K. (2023) · Peptide Science · doi:10.1002/pep2.24307 · Cited by 18
Super High-Throughput Screening of Enzyme Variants by Spectral Graph Convolutional Neural Networks
Ramírez-Palacios, C., & Marrink, . J. (2023) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c01227 · Cited by 18
DeCOIL: Optimization of Degenerate Codon Libraries for Machine Learning-Assisted Protein Engineering
Yang, J., Ducharme, J., Johnston, K. E., Li, F.-Z., Yue, Y., & Arnold, F. H. (2023) · ACS Synthetic Biology · doi:10.1021/acssynbio.3c00301 · Cited by 20
Simplifying complex antibody engineering using machine learning
Makowski, E. K., Chen, H.-T., & Tessier, P. M. (2023) · Cell Systems · doi:10.1016/j.cels.2023.04.009 · Cited by 19
An Effective Plant Small Secretory Peptide Recognition Model Based on Feature Correction Strategy
Wang, R., Zhou, Z., Wu, X., Jiang, X., Zhuo, L., Liu, M., Li, H., Fu, X., & Yao, X. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00868 · Cited by 18
Praljak, N., Lian, X., Ranganathan, R., & Ferguson, A. L. (2023) · ACS Synthetic Biology · doi:10.1021/acssynbio.3c00261 · Cited by 18
Wang, H., Hao, X., He, Y., & Fan, L. (2024) · PLoS ONE · doi:10.1371/journal.pone.0296737 · Cited by 18
“…sequences as sentences: some Pre-trained Protein Language Models (PPLMs) such as ESM [23], ProtBERT [24], ProtT5-XL-UniRef50 [24], ProtNLM [25], Unirep [26], and Pre-trained Antibody Language Models (PALMs) such as AntiBERTy [27], AntiBERTa [28], and EATLM [29] have been trained [29, 30].”
MethodDeep learning in template-free de novo biosynthetic pathway design of natural products
Xie, X., Gui, L., Qiao, B., Wang, G., Huang, S., Zhao, Y., & Sun, S. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae495 · Cited by 21
“UniRep has adopted a similar approach but integrates a ridge regression model on top of its language model [116].”
MethodPrediction of mutation effects using a deep temporal convolutional network
Kim, H. Y., & Kim, D. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz873 · Cited by 18
Deep protein representations enable recombinant protein expression prediction
Martiny, H.-M., Armenteros, J. J. A., Johansen, A. R., Salomon, J., & Nielsen, H. (2021) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2021.107596 · Cited by 16
Direct coupling analysis of epistasis in allosteric materials
Bravi, B., Ravasio, R., Brito, C., & Wyart, M. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007630 · Cited by 21
End-to-end multitask learning, from protein language to protein features without alignmentsPreprint
Elnaggar, A., Heinzinger, M., Dallago, C., & Rost, B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/864405 · Cited by 17
Fast and flexible design of novel proteins using graph neural networksPreprint
Strokach, A. V., Becerra, D., Corbi‐Verge, C., Perez-Riba, A., & Kim, P. M. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/868935 · Cited by 18
Mixture of Experts Enable Efficient and Effective Protein Understanding and DesignPreprint
Sun, N., Zou, S., Tao, T., Mahbub, S., Li, D., Zhuang, Y., Wang, H., Cheng, X., Song, L., & Xing, E. P. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.29.625425 · Cited by 26
“The huge success of large language models (LLMs) in natural language processing (NLP) inspire researchers to apply self-supervised pre-training in the protein domain, using protein sequences without any structural and functional labels [5, 6, 7, 8, 9, 10, 11, 12, 13, 3, 14, 15, 4].”
AAontology: An Ontology of Amino Acid Scales for Interpretable Machine Learning
Breimann, S., Kamp, F., Steiner, H., & Frishman, D. (2024) · Journal of Molecular Biology · doi:10.1016/j.jmb.2024.168717 · Cited by 15
Advances in Machine Learning Models for Predicting Enzyme Kinetic Parameters
Malli, A., Vasyutyn, D., & Kim, J. R. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c02428 · Cited by 19
Wang, L., Zeng, Z., Xue, Z., & Wang, Y. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.12.004 · Cited by 18
“Recently, different researchers[25-28] have developed some deep language models based on large-scale protein sequence datasets.”
BackgroundLearning the shape of protein microenvironments with a holographic convolutional neural network
Pun, M. N., Ivanov, A., Bellamy, Q., Montague, Z., LaMont, C., Bradley, P. H., Otwinowski, J., & Nourmohammad, A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2300838121 · Cited by 18
Huang, Z., Chen, S., He, K., Yu, T., Fu, J., Gao, S., & Li, H. (2024) · Horticulture Research · doi:10.1093/hr/uhae082 · Cited by 18
Enabling high-throughput enzyme discovery and engineering with a low-cost, robot-assisted pipeline
Norton‐Baker, B., Denton, M. C. R., Murphy, N. P., Fram, B. J., Lim, S., Erickson, E., Gauthier, N. P., & Beckham, G. T. (2024) · Scientific Reports · doi:10.1038/s41598-024-64938-0 · Cited by 17
Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs
Mezentsev, G., Gusak, D., Oseledets, I. V., & Frolov, E. (2024) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3640457.3688140 · Cited by 31
Wang, T., Liang, C., Hou, Y., Zheng, M., Xu, H., An, Y., Xiao, S., Liu, L., & Lian, S. (2020) · Biotechnology Letters · doi:10.1007/s10529-020-02914-0 · Cited by 17
Deep Learning for Protein–Protein Interaction Site Prediction
Jamasb, A. R., Day, B., Cangea, C., Lió, P., & Blundell, T. L. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1641-3_16 · Cited by 19
Lim, H., Jeon, H.-N., Lim, S., Jang, Y., Kim, T., Cho, H., Pan, J., & No, K. T. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.01.027 · Cited by 17
“UniRep may connote fundamental features of protein sequences because clustering with UniRep can distinguish bio-physical single amino acid properties, secondary structural helix-sheet properties, and evolutional proteome properties [4].”
BackgroundMethodInfluentialJing, R., Xue, L., Li, M., Yu, L., & Luo, J. (2022) · iScience · doi:10.1016/j.isci.2022.105530 · Cited by 17
Language modelling for biological sequences – curated datasets and baselinesPreprint
Armenteros, J. J. A., Johansen, A. R., Winther, O., & Nielsen, H. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.03.09.983585 · Cited by 19
Protein Structural Alignments From SequencePreprint
Morton, J. T., Strauss, C. E. M., Blackwell, R., Berenberg, D., Gligorijević, V., & Bonneau, R. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.11.03.365932 · Cited by 17
Drug-Target Binding Affinity Prediction Using TransformersPreprint
Saadat, M., Behjati, A., Zare‐Mirakabad, F., & Gharaghani, S. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.09.30.462610 · Cited by 17
“• UniRep: UniRep[13] is a pre-trained model that uses a multiplicative long short term memory (mLSTM) to learn protein representation.”
MethodInfluentialA Deep SE(3)-Equivariant Model for Learning Inverse Protein FoldingPreprint
McPartlon, M., Lai, B., & Xu, J. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.04.15.488492 · Cited by 20
“Moreover, our model outperforms protein language models TAPE (Rao et al. 2019) UniRep (Alley et al. 2019) trained on large sequence databases across 12 DMS datasets in zero-shot protein mutation effect prediction regardless of sequence evolutionary information as shown in Figure 3(A).”
MethodTM-Vec: template modeling vectors for fast homology detection and alignmentPreprint
Hamamsy, T., Morton, J. T., Berenberg, D., Carriero, N. J., Gligorijević, V., Blackwell, R., Strauss, C. E. M., Leman, J. K., Cho, K., & Bonneau, R. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.07.25.501437 · Cited by 16
“Multiple studies have shown the merits of these models when performing protein structure prediction, remote homology and protein design [29, 55, 28, 31, 30, 32, 33].”
BackgroundFoundation models of protein sequences: A brief overview
Bjerregaard, A., Groth, P. M., Hauberg, S., Krogh, A., & Boomsma, W. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103004 · Cited by 16
Predicting pathogen evolution and immune evasion in the age of artificial intelligence
Hamelin, D. J., Scicluna, M., Saadie, I., Mostefai, F., Grenier, J., Baron, C., Caron, É., & Hussin, J. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.03.044 · Cited by 21
How artificial intelligence is reengineering protein engineering
Listgarten, J., & Jiang, H. (2026) · Science · doi:10.1126/science.aec8444 · Cited by 20
Protein sequence profile prediction using ProtAlbert transformer
Behjati, A., Zare‐Mirakabad, F., Arab, S. S., & Nowzari-Dalini, A. (2022) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2022.107717 · Cited by 22
Recent developments of sequence-based prediction of protein–protein interactions
Murakami, Y., & Mizuguchi, K. (2022) · Biophysical Reviews · doi:10.1007/s12551-022-01038-1 · Cited by 16
Mardikoraem, M., & Woldring, D. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15051337 · Cited by 16
“introduced UniRep [32], a deep learning model that was trained on 24 million unique protein sequences to perform the next amino acid prediction tasks for extracting information about the global fitness landscape of proteins.”
BackgroundMethodPan, Y., Zhang, Y., Zhang, J., & Lu, M. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04977-8 · Cited by 16
Generative adversarial networks in protein and ligand structure generation: a case study
Faizi, S. A. A., Singh, N. K., Kamal, A., & Raza, K. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-22299-3.00014-1 · Cited by 16
Sequence-guided protein structure determination using graph convolutional and recurrent networks
Li, P.-N., de Oliveira, S. H. P., Wakatsuki, S., & van den Bedem, H. (2020) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe50027.2020.00028 · Cited by 16
Application of Sequence Embedding in Protein Sequence-Based Predictions
Ibtehaz, N., & Kihara, D. (2022) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789811258589_0002 · Cited by 18
A comprehensive survey of genome language models in bioinformatics
Shu, L., Tang, J., Guan, X., & Zhang, D. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf724 · Cited by 22
“Inspired by these successes, LLM-based frameworks have catalyzed breakthroughs in bioinformatics, particularly in protein structure prediction through models like AlphaFold [22], signaling a paradigm shift toward sequence-based modeling, and representation learning in biology [23–27].”
Metadata
- Authors
- Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M.
- Affiliations
- Harvard University (US)
- Corresponding author
- Church, George M.
- Venue
- Nature Methods
- Published
- 2019-10-21
- Volume
- 16
- Issue
- 12
- Pages
- 1315-1322
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC7067682
- OpenAlex
- W2980789587
- Cited by
- 1,131
- References
- 80
- Citations by year
- 2026: 130 · 2025: 170 · 2024: 201 · 2023: 197 · 2022: 188 · 2021: 159 · 2020: 69 · 2019: 17
- FWCI
- 35.71
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Nature Methods (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- Funders
- Amazon Web Services Cloud Credits for Research
- Center for Effective Altruism
- U.S. Department of Health & Human Services | National Institutes of Health (T32HG002295, U54-CA225088)
- National Science Foundation GRFP Fellowship
- U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (P50GM107618)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- rational protein design, protein stability prediction, de novo protein design, protein mutants, recurrent neural networks, protein function prediction
- MeSH terms
- Deep Learning, Amino Acid Sequence, Mutation, Protein Engineering, Protein Stability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
