Cited by 1,131Protein Structure and DynamicsMachine Learning in BioinformaticsRNA and protein synthesis mechanisms

The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.

Unified rational protein engineering with sequence-based deep representation learning

Open access

Alley, Ethan C., Khimulya, Grigory, Biswas, Surojit, AlQuraishi, Mohammed, Church, George M.·2019·Nature Methods

DOI 10.1038/s41592-019-0598-1PMC PMC7067682

Published version of a preprint

Open this paper in CoChat — free account Open PDF

Abstract

Rational protein engineering requires a holistic understanding of protein function. Here, we apply deep learning to unlabeled amino-acid sequences to distill the fundamental features of a protein into a statistical representation that is semantically rich and structurally, evolutionarily and biophysically grounded. We show that the simplest models built on top of this unified representation (UniRep) are broadly applicable and generalize to unseen regions of sequence space. Our data-driven approach predicts the stability of natural and de novo designed proteins, and the quantitative function of molecularly diverse mutants, competitively with the state-of-the-art methods. UniRep further enables two orders of magnitude efficiency improvement in a protein engineering task. UniRep is a versatile summary of fundamental protein features that can be applied across protein engineering informatics.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1038/s41592-019-0598-1

Cites

80 of 80 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We used basic agglomerative clustering (implemented in Python with sklearn 73 ) with the average linkage metric and Euclidean distance for all vector representations, and a precomputed Levenshtein distance matrix using the Python Levenshtein package for the sequence-only control.”
Method
The Protein Data Bank
Berman, H. M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.235 · Cited by 40,196 · cited 4×
“We computed the full sequence of hidden states at every position of 3,720 single-domain proteins from the SCOP database 60 (for which we had secondary structure annotation for each of the positions obtained from the PDB 80 ).”
MethodInfluential
Untitled work
(2008) · Journal of Machine Learning Research · Cited by 33,051
Dictionary of protein secondary structure: Pattern recognition of hydrogen‐bonded and geometrical features
Kabsch, W., & Sander, C. (1983) · Biopolymers · doi:10.1002/bip.360221211 · Cited by 15,320
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 6,122
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
Valiev, M., Bylaska, E. J., Govind, N., Kowalski, K., Straatsma, T. P., van Dam, H. J. J., Wang, D., Nieplocha, J., Aprà, E., Windus, T. L., et al. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2010.04.018 · Cited by 5,849
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 3,827
Molecular dynamics simulations of biomolecules
Karplus, M., & McCammon, J. (2002) · Nature Structural Biology · doi:10.1038/nsb0902-646 · Cited by 3,401

Show all 80 references

Research works printed but not linked: 8

Brookes, D. H., Park, H. & Listgarten, J. Conditioning by adaptive sampling for robust design. Proc. Machine Learn. Res. 97, 773–782 (2019).
Hernández-Lobato, J. M., Requeima, J., Pyzer-Knapp, E. O. & Aspuru-Guzik, A. Parallel and distributed thompson sampling for large-scale accelerated exploration of chemical space. Preprint at arXiv https://arxiv.org/abs/1706.01825 (2017). · arXiv
Snoek, J., Larochelle, H. & Adams, R. P. in Advances in Neural Information Processing Systems Vol. 25 (eds. Pereira, F. et al.) 2951–2959 (Curran Associates, Inc., 2012).
Griffiths, R.-R. & Hernández-Lobato, J. M. Constrained Bayesian optimization for automaticchemical design. Preprint at arXiv https://arxiv.org/abs/1709.05501 (2017). · arXiv
Yang, K. K., Chen, Y., Lee, A. & Yue, Y. Batched stochastic Bayesian optimization via combinatorial constraints design. Preprint at arXiv https://arxiv.org/abs/1904.08102 (2019). · arXiv
González, J., Longworth, J., James, D. C. & Lawrence, N. D. Bayesian optimization for synthetic gene design. Preprint at arXiv https://arxiv.org/abs/1505.01627 (2015). · arXiv
Jouppi, N. P. et al. In-datacenter performance analysis of a tensorprocessing unit. In Proc. 44th Annual International Symposium of Computer Architecture Vol. 45, 1–12 (ACM, 2017).
Salimans, T. & Kingma, D. P. Weight normalization: a simple reparameterization to accelerate training of deep neural networks. Preprint at arXiv https://arxiv.org/abs/1602.07868 (2016). · arXiv

Sorted by citations

Items 701–800 of 1,003

SPDesign: protein sequence designer based on structural sequence profile using ultrafast shape recognition
(2024) · doi:10.1093/bib/bbae146 · Cited by 6
Genotypic–phenotypic landscape computation based on first principle and deep learning
Liu, Y., Luo, Y., Lu, X., Gao, H., He, R., Zhang, X., Zhang, X., & Li, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae191 · Cited by 6
“In recent years, natural language models have made great strides in learning the composition rules of DNA and amino acid sequences and exploring the evolution of biological species [3, 9– 11].”
Background
TooT-BERT-T: A BERT Approach on Discriminating Transport Proteins from Non-transport Proteins
Ghazikhani, H., & Butler, G. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-17024-9_1 · Cited by 6
Bioactivity descriptors for uncharacterized compoundsPreprint
Bertoni, M., Duran‐Frigola, M., Badia-i-Mompel, P., Pauls, E., Orozco, M., Guitart-Pla, O., Alcalde, V., Diaz, V. M., Berenguer, A., de Herreros, A. G., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.21.214197 · Cited by 6
The structure-fitness landscape of pairwise relations in generative sequence modelsPreprint
Marshall, D., Wang, H., Stiffler, M. A., Dauparas, J., Koo, P. K., & Ovchinnikov, S. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.11.29.402875 · Cited by 8
“…saliency, utilizing the Hessian towards distinguishing triwise decomposable relations, and perhaps even applying it to large natural language processing based protein sequence models that are currently in vogue [Rao et al., 2019, Alley et al., 2019, Rives et al., 2019, Elnaggar et al., 2020].”
Method
Evolution Is All You Need: Phylogenetic Augmentation for Contrastive LearningPreprint
Lu, A. X., Lu, A. X., & Moses, A. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13475 · Cited by 17
Benchmarking deep generative models for diverse antibody sequence designPreprint
Melnyk, I., Das, P., Chenthamarakshan, V., & Lozano, A. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06801 · Cited by 19
“Alltrainingsequencesarepadded N s of200,as77%ofsingle-domain For example, the sequence-based methods of [2, 13, 1] formulated the problem as an autoregressive process, leveraging deep recurrent networks such as an LSTM, to design the entire antibody sequence.”
Method
Regression Transformer enables concurrent sequence regression and generation for molecular language modellingPreprint
Born, J., & Manica, M. (2022) · arXiv (Cornell University) · doi:10.3929/ethz-b-000609688 · Cited by 6
Learning the shape of protein micro-environments with a holographic convolutional neural networkPreprint
Pun, M. N., Ivanov, A., Bellamy, Q., Montague, Z., LaMont, C., Bradley, P. H., Otwinowski, J., & Nourmohammad, A. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.10.31.514614 · Cited by 7
Likelihood-based Fine-tuning of Protein Language Models for Few-shot Fitness Prediction and DesignPreprint
Hawkins‐Hooker, A., Surana, S., Simons, J., Kmec, J., Bent, O., & Duckworth, P. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.28.596156 · Cited by 10
“One popular paradigm for exploiting the information in pretrained PLMs involves extracting sequence representations and feeding these as inputs into task-specific down-stream predictive models (Alley et al., 2019; Biswas et al., 2021; Rao et al., 2019; Dallago et al., 2021; Notin et al., 2023b).”
Benchmarking gene embeddings from sequence, expression, network, and text models for functional prediction tasksPreprint
Zhong, J., Li, L., Dannenfelser, R., & Yao, V. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.01.29.635607 · Cited by 9
“…BERT-BFD [23], BERT-PFAM [37], BLAST [8], CPC-PROT [38], ESMB1 [26], GENE2VEC [30], HMMER [9], KSEP [13], LEARNED-VEC [21], MUT2VEC [32], PFAM [9], PROTVEC [22], SEQVEC [19], T5 [23], TCGA-EMBEDDING [29], UNIREP [20], and XLNET [23] were sourced from the Protein RepresentatiOn BEnchmark study [7].”
Deconvolving mutation effects on protein stability and function with disentangled protein language modelsPreprint
Ding, K., Li, Z., Tu, T., Luo, J., & Luo, Y. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.03.703560 · Cited by 7
“The changes in log-likelihood between mutant and wild type have been observed to correlate well with diverse MAVE readouts 38–42 .”
Intrinsic dataset features drive mutational effect prediction by protein language modelsPreprint
Vieira, L. C., Lin, S., & Wilke, C. O. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.08.710389 · Cited by 12
“During pretraining, these models learn internal representations, referred to as embeddings, that encode information about protein biochemistry, structure, and evolution (6, 7).”
MM-DRPNet: A multimodal dynamic radial partitioning network for enhanced protein–ligand binding affinity prediction
Liu, D., Song, T., & Wang, S.-D. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.11.050 · Cited by 5
Differentially Private Federated Learning for Genomic-Based Stomach Adenocarcinoma Detection in Consumer Medical IoT
Sikandar, M., Din, I. U., Almogren, A., & Rodrigues, J. J. P. C. (2025) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2025.3565962 · Cited by 5
“Furthermore, previous studies have demonstrated the effectiveness of LSTM in genomic research, such as cancer diagnosis [19] and protein sequence analysis [28].”
Explainability of Protein Deep Learning Models
Fazel, Z., de Souza, C. P. E., Golding, G. B., & Ilie, L. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26115255 · Cited by 5
AOPxSVM: A Support Vector Machine for Identifying Antioxidant Peptides Using a Block Substitution Matrix and Amino Acid Composition, Transformation, and Distribution Embeddings
Li, R., Wang, H., Yu, Q. P., Cai, J., Jiang, L., Luo, X., Zou, Q., & Lv, Z. (2025) · Foods · doi:10.3390/foods14122014 · Cited by 5
“UniRep [42] is constructed based on a Multiplicative Long Short-term Memory (mL-STM), and unsupervised learning is performed on the UniRef50 [43] database containing 24 million proteins.”
A Three-Module Machine Learning Framework for Protein Sequence- and Temperature-Dependent kcat/Km Prediction in β-Glucosidases
Erkanli, M. E., Jang, Y., Malli, A., El‐Halabi, K., Ryu, C., & Kim, J. R. (2025) · ACS Synthetic Biology · doi:10.1021/acssynbio.5c00257 · Cited by 5
ALLM-Ab: Active Learning-Driven Antibody Optimization Using Fine-Tuned Protein Language Models
Furui, K., & Ohue, M. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01577 · Cited by 5
“In the offline active learning experiments, although a conventional latent space-based Gaussian process regression (GPR) approach exhibited higher predictive performance, it struggled in the online setting due to poor extrapolation to out-of-distribution data.”
PBIP: a deep learning framework for predicting phage–bacterium interactions at the strain level
Ma, L., Gao, P., Liu, G., Bai, Y., Lin, Q., Li, J., & Xiao, M. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf656 · Cited by 6
“To overcome this limitation, we employ UniRep [39], a pretrained protein representation model that encodes sequences into 1900-dimensional embeddings while preserving physico-chemical and structural properties.”
Rapid customization of base editors via machine learning-powered combinatorial mutagenesis
Peng, J., Chan, D. C. T., Chu, H. Y., Fong, J. H. C., Lam, Y. K., Cheung, M. S. H., Leung, W., Choi, G. C. G., & Wong, A. S. L. (2026) · Molecular Cell · doi:10.1016/j.molcel.2026.03.030 · Cited by 5
PseAAC2Vec protein encoding for TCR protein sequence classification
Tayebi, Z., Ali, S., Murad, T., Khan, I., & Patterson, M. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107956 · Cited by 6
ThermoFinder: A sequence-based thermophilic proteins prediction framework
Yu, H., & Luo, X. (2024) · International Journal of Biological Macromolecules · doi:10.1016/j.ijbiomac.2024.132469 · Cited by 5
Jointly Embedding Protein Structures and Sequences through Residue Level Alignment
Birnbaum, F., Jain, S., Mądry, A., & Keating, A. E. (2024) · PRX Life · doi:10.1103/prxlife.2.043013 · Cited by 5
“Several groups have developed large language model encoders of protein sequences [6– 8].”
PTSP-BERT: Predict the thermal stability of proteins using sequence-based bidirectional representations from transformer-embedded features
Lv, Z., Wei, M., Pei, H., Peng, S., Li, M., & Jiang, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109598 · Cited by 5
MProt-DPO: Breaking the ExaFLOPS Barrier for Multimodal Protein Design Workflows with Direct Preference Optimization
Dharuman, G., Hippe, K., Brace, A., Foreman, S., Hatanpää, V., Sastry, V. K., Zheng, H., Ward, L. T., Muralidharan, S., Vasan, A. K., et al. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41406.2024.00013 · Cited by 9
“PLMs have also been successfully applied to generative modeling for protein design [8], [9], [21], protein function classification [22], and other downstream tasks [23].”
SST-ResNet: A Sequence and Structure Information Integration Model for Protein Property Prediction
Zhou, G., Zhao, Y., He, S., & Bo, X. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26062783 · Cited by 5
“Prominent examples of protein language models (PLMs) are primarily transformer-based, such as ESM [11], as well as long short-term memory (LSTM)-based LMs from Alley et al. [12] and Heinzinger et al. [13].”
Artificial intelligence and first‐principle methods in protein redesign: A marriage of convenience?
Cianferoni, D., Vizarraga, D., Fernández‐Escamilla, A. M., Fita, I., Hamdani, R., Reche, R., Delgado, J., & Serrano, L. (2025) · Protein Science · doi:10.1002/pro.70210 · Cited by 4
“Recent advancements in machine learning (Alley et al. 2019; Wang et al. 2017; Zheng et al. 2019) have fueled the potential of de novo design, which has been demonstrated to be highly effective in generating novel molecules with desired functions (Anand et al. 2022; Anishchenko et al. 2021).”
Current status and opportunities in adaptive data analysis for therapeutic cell manufacturing
Liu, Z., Chen, J., Wang, K., Wang, B., & Zhang, C. (2021) · Current Opinion in Biomedical Engineering · doi:10.1016/j.cobme.2021.100351 · Cited by 5
Data-driven design of orthogonal protein-protein interactions
Malinverni, D., & Babu, M. M. (2023) · Science Signaling · doi:10.1126/scisignal.abm4484 · Cited by 7
Progress and challenges for the application of machine learning for neglected tropical diseases
Khew, C., Akbar, R., & Mohd‐Assaad, N. (2023) · F1000Research · doi:10.12688/f1000research.129064.1 · Cited by 5
“…employ evolutionary information from MSA inputs, substituting sequence similarity for homology-based annotation transfer, and predicting mutational effects on protein-protein interactions (PPI) (Alley et al. 2019; Heinzinger et al. 2019; Littmann et al. 2021; Stärk et al. 2021; Zhou et al. 2020).”
Influential
CysPresso: a classification model utilizing deep learning protein representations to predict recombinant expression of cysteine-dense peptides
Ouellet, S., Ferguson, L., Lau, A. Z., & Lim, T. K. Y. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05327-8 · Cited by 5
“As this model was the closest known comparable model to CysPresso, we obtained UniRep representations [28] for the dataset utilized in this study and evaluated the performance of UniRep-RF and CysPresso using leave-oneout cross-validation (Table 3).”
BackgroundMethod
iNP_ESM: Neuropeptide Identification Based on Evolutionary Scale Modeling and Unified Representation Embedding Features
Li, H., Jiang, L., Yang, K.-X., Shang, S.-L., Li, M., & Lv, Z.-B. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25137049 · Cited by 4
“The formula for calculating the similarity score is as follows [44]: where z i and z ′ j are the embedding vectors of two protein sequences, a ij is an element of the alignment matrix representing the alignment weight between positions i and j of the sequences, and A is the total length of the…”
BackgroundMethodInfluential
Deep Learning-Based Self-Adaptive Evolution of Enzymes
Jiang, S., & Yi, D. (2024) · Pharmaceutical Fronts · doi:10.1055/s-0044-1788317 · Cited by 5
ProteinReDiff: Complex-based ligand-binding proteins redesign by equivariant diffusion-based generative models
Nguyen, V. T. D., Nguyen, N. D., & Hy, T. S. (2024) · Structural Dynamics · doi:10.1063/4.0000271 · Cited by 8
Web tools support predicting protein–nucleic acid complexes stability with affinity changes
Zhang, X. L., Mei, L., Gao, Y., Hao, G., & Song, B. (2023) · Wiley Interdisciplinary Reviews - RNA · doi:10.1002/wrna.1781 · Cited by 5
“Protein engineering modifications affect protein adsorption, ligand binding, or protein complex thermodynamic stability (Alley et al., 2019).”
Background
Generating interacting protein sequences using domain-to-domain translation
Meynard-Piganeau, B., Fabbri, C., Weigt, M., Pagnani, A., & Feinauer, C. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad401 · Cited by 6
“2021), LSTM-based architectures (Alley et al. 2019), and selfattention-based architectures (Madani et al.”
Background
Learning to Make Decisions via Submodular Regularization
Alieva, A., Aceves, A. J., Song, J., Mayo, S. L., Yue, Y., & Chen, Y. (2021) · International Conference on Learning Representations · Cited by 5
Deep Learning for RNA Synthetic BiologyPreprint
Angenent-Mari, N. M., Garruss, A. S., Soenksen, L. R., Church, G. M., & Collins, J. J. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/872077 · Cited by 5
Self-citation
Evaluating eUniRep and other protein feature representations for in silico directed evolutionPreprint
Favor, A. H., & Jayapurna, I. (2020) · doi:10.22541/au.159683529.96283070 · Cited by 5
LOGO, a contextualized pre-trained language model of human genome flexibly adapts to various downstream tasks by fine-tuningPreprint
Yang, M., Huang, H., Huang, L., Zhang, N., Wu, J., Yang, H., & Mu, F. (2021) · Research Square · doi:10.21203/rs.3.rs-448927/v1 · Cited by 5
Sharing to learn and learning to share; Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta reviewPreprint
Upadhyay, R., Phlypo, R., Saini, R., & Liwicki, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12146 · Cited by 5
Drug repurposing for Leishmaniasis with Hyperbolic Graph Neural NetworksPreprint
Lau, Y., Gutierrez, J. M., Volkovs, M., & Zuberi, S. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.11.528117 · Cited by 7
“We used UniRep [27] to extract features in the form of 1,900-dimensional vectors for proteins.”
Method
Retrieved Sequence Augmentation for Protein Representation LearningPreprint
Ma, C., Zhao, H., Zheng, L., Xin, J., Li, Q., Wu, L., Deng, Z., Yang, L., Liu, Q., & Kong, L. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.22.529597 · Cited by 7
“Protein Language Models To model and further understand the protein sequence data, language models are introduced to train on mass data (Heinzinger et al., 2019; Alley et al., 2019).”
Method
Assessing the performance of protein regression modelsPreprint
Richard, M., Kæstel‐Hansen, J., Groth, P. M., Bartels, S., Salomon, J., Tian, P., Hatzakis, N. S., & Boomsma, W. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.06.18.545472 · Cited by 5
“Advances in representation learning have shown that different approaches to unsupervised learning from biological sequences lead to differently structured representations (8;12;13;24;25;20).”
BackgroundMethod
TRILL: Orchestrating Modular Deep-Learning Workflows for Democratized, Scalable Protein Analysis and EngineeringPreprint
Martinez, Z. A., Murray, R. M., & Thomson, M. W. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.24.563881 · Cited by 7
“Deep learning methods, such as recurrent neural networks, have proven to be able to extract functional information of proteins from sequence alone [9].”
Background
Neural network extrapolation to distant regions of the protein fitness landscapePreprint
Fahlberg, S. A., Freschlin, C. R., Heinzelman, P., & Romero, P. A. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.08.566287 · Cited by 5
Pseudo-perplexity in one fell swoop for protein fitness estimation
Kantroo, P., Wagner, G. P., & Machta, B. B. (2025) · PRX Life · doi:10.1103/zhx7-hcmm · Cited by 8
ARNLE model identifies prevalence potential of SARS-CoV-2 variants
Liu, Y., Li, J., Li, P., Wang, K., Wang, K., Li, J., Yang, L., Liu, J., Jia, L., Wu, A., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00919-2 · Cited by 4
“Multiplicative and bidirectional long-short-term-memory (multi-LSTM and bi-LSTM) DL techniques can semantically and grammatically provide more information about protein structure, evolution and function 27,28 .”
Influential
A roadmap for T cell receptor-peptide-bound major histocompatibility complex binding prediction by machine learning: glimpse and foresight
Qi, F., Huang, Q., Yao, X., Cao, Y., Shen, Y., Ren, Y., Liu, Z., & Zhang, Z. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf327 · Cited by 4
“Pretrained protein language models include UniRep [49], ESM [50], ProtTrans [51], Progen [52], ProtGPT2 [53], and xTrimoPGLM [54].”
A Benchmarking Platform for Assessing Protein Language Models on Function-Related Prediction Tasks
Çevrim, E., Yiğit, M. G., Ulusoy, E., Yılmaz, A., & Doğan, T. (2025) · Methods in molecular biology · doi:10.1007/978-1-0716-4662-5_14 · Cited by 7
An iterative deep learning-guided algorithm for directed protein evolution
Li, X., Wang, Q., Li, J., Tang, F., Yang, X., & Lin, Z. (2025) · iScience · doi:10.1016/j.isci.2025.113324 · Cited by 4
“In the process of the avGFP design, we used the previously described evotuned weights 19 and repeated the evotuning process to ensure its robustness.”
Biotechnology meets AI: Accelerating next-generation of microbial breeding for food biomanufacturing
Tan, Y., Ni, H., Wu, T., Li, W., Zeng, H., & Wang, Y. (2025) · Food Research International · doi:10.1016/j.foodres.2025.118152 · Cited by 4
Advances in Prediction of Posttranslational Modification Sites Known to Localize in Protein Supersecondary Structures
Pratyush, P., & KC, D. B. (2024) · Methods in molecular biology · doi:10.1007/978-1-0716-4213-9_8 · Cited by 4
Reprogramming pretrained language models for protein sequence representation learning
Vinod, R., Chen, P., & Das, P. (2025) · Digital Discovery · doi:10.1039/d4dd00195h · Cited by 17
iAVP-RFVOT: Identify Antiviral Peptides by Random Forest Voting Machine Learning with Unified Manifold Learning Embedded Features
Wang, H., Li, R., Yu, Q. P., Jiang, L., Luo, X., Zou, Q., & Lv, Z. (2025) · Biochemistry · doi:10.1021/acs.biochem.5c00237 · Cited by 4
Tracking biomedical articles along the translational continuum: a measure based on biomedical knowledge representation
Li, X., Tang, X., & Lu, W. P. (2022) · Scientometrics · doi:10.1007/s11192-022-04607-z · Cited by 4
“1 3 on titles and abstracts have been successfully used for solving multiple tasks in the field of biomedicine (Alley et al., 2019; Chen et al., 2019; El-allay et al., 2021).”
Method
DeepSub: Utilizing Deep Learning for Predicting the Number of Subunits in Homo-Oligomeric Protein Complexes
Deng, R., Wu, K., Lin, J., Wang, D., Huang, Y., Li, Y., Shi, Z., Zhang, Z., Wang, Z., Mao, Z., et al. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25094803 · Cited by 4
“Firstly, semantic representations of protein sequences are obtained through ESM2 [22], which is an advanced protein language model aimed at understanding and predicting the structure and function of proteins [35].”
Method
ACP-PDAFF: Pretrained model and dual-channel attentional feature fusion for anticancer peptides prediction
Wang, X., & Wang, S. (2024) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2024.108141 · Cited by 4
CCPL: Cross-Modal Contrastive Protein Learning
Zheng, J., & Li, S. Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78398-2_2 · Cited by 5
“Models such as UniRep [1] utilize LSTM or its variants to capture sequence representations and long-range dependencies.”
Method
CBKG-DTI: Multi-Level Knowledge Distillation and Biomedical Knowledge Graph for Drug-Target Interaction Prediction
Zhao, X., Wang, Q., Zhang, Y., He, C., Yin, M., & Zhao, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3500027 · Cited by 4
TMVisDB: Annotation and 3D-visualization of Transmembrane Proteins
Olenyi, T., Marquet, C., Grekova, A., Houri, L., Heinzinger, M., Dallago, C., & Rost, B. (2025) · Journal of Molecular Biology · doi:10.1016/j.jmb.2025.168997 · Cited by 4
LatentDE: latent-based directed evolution for protein sequence design
Tran, T. P., Ngo, N. K., Nguyen, V. T. D., & Hy, T. S. (2025) · Machine Learning Science and Technology · doi:10.1088/2632-2153/adc2e2 · Cited by 6
Deep Learning Proteins using a Triplet-BERT network
Lennox, M., Robertson, N. M., & Devereux, B. J. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9630387 · Cited by 4
“In addition to these three models, they also benchmarked two previously proposed architectures; another a bidirectional LSTM model [2], and a unidirectional mLSTM [20, 1].”
Method
Density Peak clustering of protein sequences associated to a Pfam clan reveals clear similarities and interesting differences with respect to manual family annotation
Russo, E. T., Laio, A., & Punta, M. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04013-x · Cited by 4
Navigating the amino acid sequence space between functional proteins using a deep learning framework
Bitard‐Feildel, T. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.684 · Cited by 4
“Recent advances have been made regarding the protein sequence universe representation notably using self-supervised approaches, notably with the Transformer architecture (Alley et al., 2019; Heinzinger et al., 2019; Rao et al., 2019; Rives et al., 2019; Strodthoff et al., 2020).”
Method
Machine Learning-driven Protein Library Design: A Path Toward Smarter Libraries
Mardikoraem, M., & Woldring, D. (2022) · Methods in molecular biology · doi:10.1007/978-1-0716-2285-8_5 · Cited by 4
Organizing the bacterial annotation space with amino acid sequence embeddings
Grigson, S. R., McKerral, J. C., Mitchell, J. G., & Edwards, R. A. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04930-5 · Cited by 4
“These learning abilities indicate that sequence embeddings can be generalized across all protein sequences to infer and relate protein functions [21, 46].”
Background
Sensing the shape of functional proteins with topology
Luo, Y. (2023) · Nature Computational Science · doi:10.1038/s43588-023-00404-7 · Cited by 4
BERT2DAb: a pre-trained model for antibody representation based on amino acid sequences and 2D-structure
Luo, X., Tong, F., Zhao, W., Zheng, X., Li, J., Li, J., & Zhao, D. (2023) · mAbs · doi:10.1080/19420862.2023.2285904 · Cited by 4
“14 ESM2, ProtTrans, and UniRep are examples of PLM. 13,15,16 Among these models, the first two are based on self-attention and include multiple models with varying parameter sizes, while the last one is a single model based on LSTM.”
Background
Prediction of Protein Tertiary Structure Using Pre-Trained Self-Supervised Learning Based on Transformer
Kurniawan, A., Jatmiko, W., Hertadi, R., & Habibie, N. (2020) · International Workshop on Big Data and Information Security (IWBIS) · doi:10.1109/iwbis50925.2020.9255624 · Cited by 4
Augmenting protein network embeddings with sequence informationPreprint
Kané, H., Coulibali, M., Abdalla, A., & Ajanoh, P. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/730481 · Cited by 3
Papyrus - A large scale curated dataset aimed at bioactivity predictionsPreprint
Béquignon, O. J. M., Bongers, B. J., Jespers, W., IJzerman, A. P., van de Water, B., & van Westen, G. J. P. (2021) · doi:10.33774/chemrxiv-2021-1rxhk · Cited by 4
Comparison and evaluation of data-driven protein stability prediction modelsPreprint
Csicsery-Ronay, J. A., Zaitzeff, A., & Singer, J. M. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.03.14.483859 · Cited by 4
“We apply XGB to features generated by extracting embeddings from sequence data by the ProtBERT model [6], the evolutionary scale model (ESM) [14], and the UniRep model [1].”
BackgroundMethodInfluential
LMSuccSite: Improving Protein Succinylation Sites Prediction Using Embeddings from Protein Language ModelPreprint
Pokharel, S., Pratyush, P., Heinzinger, M., Newman, R. H., & KC, D. B. (2022) · Research Square · doi:10.21203/rs.3.rs-1953874/v1 · Cited by 4
A general prediction model for substrates of transport proteinsPreprint
Kroll, A., Niebuhr, N., Butler, G., & Lercher, M. J. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.31.564943 · Cited by 4
“In order to generate a single fixed-length numerical representation of the whole protein, one typically calculates the element-wise mean across all amino acid representations 17,36,37 .”
Background
Contrastive Fitness Learning: Reprogramming Protein Language Models for Low- N Learning of Protein Fitness LandscapePreprint
Zhao, J., Zhang, C., & Luo, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.02.11.579859 · Cited by 7
“The comparison also included eUniRep, a model specifically tailored for low-N applications 36 .”
BackgroundInfluential
CatPred: A comprehensive framework for deep learning in vitro enzyme kinetic parameters k cat , K m and K iPreprint
Boorla, V. S., & Maranas, C. D. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.03.10.584340 · Cited by 4
“They used a protein Language Model (pLM), UniRep 32 , 21 for extracting numerical representations of the enzyme and a task specific graph neural network derived fingerprints combined with the molecular mass and hydrophobicity properties as features for metabolites.”
Method
Artificial Intelligence Learns Protein PredictionPreprint
Heinzinger, M., & Rost, B. (2024) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a041458 · Cited by 4
Protein language model identifies disordered, conserved motifs implicated in phase separation
Zhang, Y., Zheng, J., & Zhang, B. (2025) · eLife · doi:10.7554/elife.105309 · Cited by 4
“Recent advances in protein language models ( Ofer et al., 2021 ; Rives et al., 2021 ; Elnaggar et al., 2021 ; Strodthoff et al., 2020 ; Alley et al., 2019 ; Brandes et al., 2022 ) provide alternative approaches for sequence analysis and information decoding.”
Strategies to identify and edit improvements in synthetic genome segments episomally
Rudolph, A., Nyerges, A., Chiappino-Pepe, A., Landon, M., Baas-Thomas, M., & Church, G. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad692 · Cited by 3
“Here, a computational tool such as UniRep that is trained on related sequences and may be used to predict the likelihood of an amino acid occurring at a particular position in a sequence could prove to be more powerful (46).”
MethodSelf-citation
Enhancing the reverse transcriptase function in Taq polymerase via AI-driven multiparametric rational design
Tomilova, Y. E., Russkikh, N., Yi, I. M., Shaburova, E. V., Tomilov, V. N., Pyrinova, G. B., Brezhneva, S. O., Tikhonyuk, O. S., Gololobova, N. S., Popichenko, D. V., et al. (2024) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2024.1495267 · Cited by 3
iNClassSec-ESM: Discovering potential non-classical secreted proteins through a novel protein language model
Shao, Y., & Liu, T. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.03.043 · Cited by 3
Multidimensional maturation of antibody variable domains with machine-learning assistance
Ito, T., Kawada, S., Nakazawa, H., Murakami, A., & Umetsu, M. (2026) · mAbs · doi:10.1080/19420862.2025.2611472 · Cited by 4
Enhancing CAR-T cell activity prediction via fine-tuning protein language models with generated CAR sequences
Yoshida, K., Hisada, S., Takase, R., Okuma, A., Ishida, Y., Kawara, T., Miura-Yamashita, T., & Ito, D. (2026) · BMC Bioinformatics · doi:10.1186/s12859-026-06401-7 · Cited by 3
Productive chaos and precision engineering: decoupling discovery from manufacturing to revolutionize plant-inspired therapeutics
Mosoh, D. A. (2026) · Frontiers in Plant Science · doi:10.3389/fpls.2026.1771802 · Cited by 3
A generative explainable model for antimicrobial peptide prediction using bidirectional temporal convolutional neural network
Ali, F., Khalid, M., Alsini, R. A. I., Yafoz, A., Alkhalifah, T., & Du, M. (2026) · Scientific Reports · doi:10.1038/s41598-026-43370-6 · Cited by 3
“The model was trained on 24 million protein sequences from the UniRef50 database to capture structural, evolutionary, and biophysical properties directly from the sequence data 34 .”
Cellulases and hemicellulases in lignocellulose degradation: A comparative analysis of wild-type and recombinant systems
Pappu, J. S. M., Kumar, K., & Kumari, R. (2026) · Biochemical and Biophysical Research Communications · doi:10.1016/j.bbrc.2026.153675 · Cited by 3
Comparative Analysis on Alignment-Based and Pretrained Feature Representations for the Identification of DNA-Binding Proteins
Chen, D., Zhang, H., Chen, Z., Xie, B., & Wang, Y. (2022) · Computational and Mathematical Methods in Medicine · doi:10.1155/2022/5847242 · Cited by 3
“ESM-1b outperforms all tested singlesequence protein language models, including UniRep [45], TAPE [46], SeqVec [47], LSTM, and Transformer, across a range of structure prediction tasks.”
Method
Evaluating the chaos game representation of proteins for applications in machine learning models: prediction of antibody affinity and specificity as a case study
Arsiccio, A., Stratta, L., & Menzen, T. (2023) · Journal of Molecular Modeling · doi:10.1007/s00894-023-05777-0 · Cited by 3
“One set of features were Unified Representation (UniRep) features, i.e., deep learning features obtained from a neural network trained on over 20 million unlabeled protein sequences to perform next amino acid prediction [24].”
Method
Investigating Enzyme Biochemistry by Deep Learning: A Computational Tool for a New Era
Rayka, M., Mirzaei, M., Farnoosh, G., & Latifi, A. M. (2024) · Journal of Computational Biophysics and Chemistry · doi:10.1142/s2737416524400052 · Cited by 3
[Translated article] Introducing artificial intelligence to hospital pharmacy departments
González-Pérez, Y., Delgado, A. M., & Sesmero, J. M. M. (2024) · Farmacia Hospitalaria · doi:10.1016/j.farma.2024.04.001 · Cited by 6
Efficiently Predicting Mutational Effect on Homologous Proteins by Evolution Encoding
Zhong, Z., & Mottin, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70368-3_24 · Cited by 3
“For this reason, protein engineering stands at the forefront of modern biotechnology, offering a remarkable toolkit to manipulate and optimise existing proteins for a wide range of applications, from drug development to personalised therapy [1].”
Background
Machine learning in molecular biophysics: Protein allostery, multi-level free energy simulations, and lipid phase transitions
Cui, Q. (2025) · Biophysics Reviews · doi:10.1063/5.0248589 · Cited by 3
$\mathcal{S}$ able: bridging the gap in protein structure understanding with an empowering and versatile pre-training paradigm
Li, J., Chen, X., Huang, H., Zeng, M., Yu, J., Gong, X., & Ye, Q. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf120 · Cited by 4
Ultrafast classical phylogenetic method beats large protein language models on variant effect prediction
Prillo, S., Wu, W., & Song, Y. S. (2024) · neural information processing systems · doi:10.52202/079017-4139 · Cited by 3
Measurements drive progress in directed evolution for precise engineering of biological systems
Tack, D. S., Romantseva, E. F., Tonner, P. D., Pressman, A., Rammohan, J., & Strychalski, E. A. (2020) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2020.09.004 · Cited by 3
Structure-based analyses of gut microbiome-related proteins by neural networks and molecular dynamics simulations
Taneishi, K., & Tsuchiya, Y. (2022) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2022.102336 · Cited by 4
inACP: An integrated approach to the prediction of anticancer peptides
Zhong, G., & Deng, L. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995361 · Cited by 3
Hosts engineering and in vitro enzymatic synthesis for the discovery of novel natural products and their derivatives
Zhang, H., Guo, L., Su, Y., Wang, R., Yang, W., Mu, W., Xuan, L., Huang, L., Wang, J., & Gao, W. (2023) · Critical Reviews in Biotechnology · doi:10.1080/07388551.2023.2236787 · Cited by 3
“…prediction of wild-type (WT) protein structures, it seemingly fails to cope with the impact of missense point mutations on protein structures [88], and unified representation (UniRep) that can predict the functional effects of different protein mutants has therefore been recently introduced [89].”

Authors: Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M.
Affiliations: Harvard University (US)
Corresponding author: Church, George M.
ORCID: Alley, Ethan C.
Biswas, Surojit
AlQuraishi, Mohammed
Church, George M.
Venue: Nature Methods
Publisher: Springer Science and Business Media LLC
Published: 2019-10-21
Volume: 16
Issue: 12
Pages: 1315-1322
ISSN: 1548-7091 (ISSN-L)
1548-7105
DOI: 10.1038/s41592-019-0598-1
PMC: PMC7067682
OpenAlex: W2980789587
Cited by: 1,131
References: 80
Citations by year: 2026: 130 · 2025: 170 · 2024: 201 · 2023: 197 · 2022: 188 · 2021: 159 · 2020: 69 · 2019: 17
FWCI: 35.71
Type: journal article
Language: en
Open access: green
doi.org
Licence: cc-by-nc-nd
www.springer.com/tdm (tdm)
www.springer.com/tdm (vor)
Locations: Nature Methods (publishedVersion)
bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
Research Square (acceptedVersion, cc-by)
PubMed (publishedVersion)
Europe PMC (PubMed Central) (acceptedVersion)
PubMed Central (acceptedVersion)
OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
Funders: Amazon Web Services Cloud Credits for Research
Center for Effective Altruism
U.S. Department of Health & Human Services | National Institutes of Health (T32HG002295, U54-CA225088)
National Science Foundation GRFP Fellowship
U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (P50GM107618)
Topics: Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
Keywords: rational protein design, protein stability prediction, de novo protein design, protein mutants, recurrent neural networks, protein function prediction
MeSH terms: Deep Learning, Amino Acid Sequence, Mutation, Protein Engineering, Protein Stability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Unified rational protein engineering with sequence-based deep representation learning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks