The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics
Open accessAbstract
We introduce a new representation and feature extraction method for biological sequences. Named bio-vectors (BioVec) to refer to biological sequences in general with protein-vectors (ProtVec) for proteins (amino-acid sequences) and gene-vectors (GeneVec) for gene sequences, this representation can be widely used in applications of deep learning in proteomics and genomics. In the present paper, we focus on protein-vectors that can be utilized in a wide array of bioinformatics investigations such as family classification, protein visualization, structure prediction, disordered protein identification, and protein-protein interaction prediction. In this method, we adopt artificial neural network approaches and represent a protein sequence with a single dense n-dimensional vector. To evaluate this method, we apply it in classification of 324,018 protein sequences obtained from Swiss-Prot belonging to 7,027 protein families, where an average family classification accuracy of 93%±0.06% is obtained, outperforming existing family classification methods. In addition, we use ProtVec representation to predict disordered proteins from structured proteins. Two databases of disordered sequences are used: the DisProt database as well as a database featuring the disordered regions of nucleoporins rich with phenylalanine-glycine repeats (FG-Nups). Using support vector machine classifiers, FG-Nup sequences are distinguished from structured protein sequences found in Protein Data Bank (PDB) with a 99.8% accuracy, and unstructured DisProt sequences are differentiated from structured DisProt sequences with 100.0% accuracy. These results indicate that by only providing sequence data for various proteins into this model, accurate information about protein structure can be determined. Importantly, this model needs to be trained only once and can then be applied to extract a comprehensive set of information regarding proteins of interest. Moreover, this representation can be considered as pre-training for various applications of deep learning in bioinformatics. The related data is available at Life Language Processing Website: http://llp.berkeley.edu and Harvard Dataverse: http://dx.doi.org/10.7910/DVN/JMFHTN.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Least Squares Support Vector Machine Classifiers
Suykens, J. A. K., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,362
Pfam: The protein families database in 2021
Mistry, J., Chuguransky, S., Williams, L., Qureshi, M., Salazar, G. A., Sonnhammer, E. L. L., Tosatto, S. C. E., Paladin, L., Raj, S., Richardson, L. J., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa913 · Cited by 7,452
Murzin, A. G., Brenner, S. E., Hubbard, T., & Chothia, C. (1995) · Journal of Molecular Biology · doi:10.1016/s0022-2836(05)80134-2 · Cited by 6,445
Pfam: the protein families database
Finn, R. D., Bateman, A., Clements, J., Coggill, P., Eberhardt, R. Y., Eddy, S. R., Heger, A., Hetherington, K., Holm, L., Mistry, J., et al. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1223 · Cited by 6,148
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Intrinsically unstructured proteins and their functions
Dyson, H. J., & Wright, P. E. (2005) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm1589 · Cited by 3,871
An efficient algorithm for large-scale detection of protein families
Enright, A. J. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.7.1575 · Cited by 3,588
Cited by
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Deep learning for computational biology
Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016) · Molecular Systems Biology · doi:10.15252/msb.20156651 · Cited by 1,473
“Skip-gram neural networks have been applied to learn low-dimensional representations of protein sequences and improve protein classification (Asgari & Mofrad, 2015).”
Deep learning in bioinformatics
Min, S., Lee, B., & Yoon, S. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw068 · Cited by 1,465
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131
Machine-learning-guided directed evolution for protein engineering
Yang, K. K., Wu, Z., & Arnold, F. H. (2019) · Nature Methods · doi:10.1038/s41592-019-0496-6 · Cited by 1,130
Automated machine learning: Review of the state-of-the-art and opportunities for healthcare
Waring, J., Lindvall, C., & Umeton, R. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101822 · Cited by 834
Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition
Jaeger, S., Fulle, S., & Turk, S. (2018) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00616 · Cited by 758
Deep Learning and Its Applications in Biomedicine
Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., Zhou, Y., Bo, X., & Xie, Z. (2018) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2017.07.003 · Cited by 624
Chen, L., Tan, X., Wang, D., Zhong, F., Liu, X., Yang, T., Luo, X., Chen, K., Jiang, H., & Zheng, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa524 · Cited by 607
Modeling the language of life – Deep Learning Protein Sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · doi:10.1101/614313 · Cited by 606
Modeling aspects of the language of life through transfer-learning protein sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3220-8 · Cited by 606
“We compared the performance of the SeqVec embeddings to state-of-the-art methods using evolutionary information, and also to a popular embedding tool for protein sequences originating from the Word2vec approach, namely ProtVec [42].”
InfluentialA deep learning genome-mining strategy for biosynthetic gene cluster prediction
Hannigan, G. D., Prihoda, D., Palicka, A., Soukup, J., Klempir, O., Rampula, L., Durcak, J., Wurst, M., Kotowski, J., Chang, D., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz654 · Cited by 413
Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review
Rasheed, K., Qayyum, A., Qadir, J., Sivathamboo, S., Kwan, P., Kuhlmann, L., O'Brien, T., & Razi, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3008792 · Cited by 409
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 339
BERTology Meets Biology: Interpreting Attention in Protein Language Models
Vig, J., Madani, A., Varshney, L. R., Xiong, C., Socher, R., & Rajani, N. F. (2020) · doi:10.1101/2020.06.26.174417 · Cited by 338
Using deep learning to annotate the protein universe
Bileschi, M. L., Belanger, D., Bryant, D. H., Sanderson, T., Carter, B., Sculley, D., Bateman, A., DePristo, M. A., & Colwell, L. J. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01179-w · Cited by 332
Predicting HLA class II antigen presentation through integrated deep learning
Chen, B., Khodadoust, M. S., Olsson, N., Wagar, L. E., Fast, E., Liu, C. L., Muftuoglu, Y., Sworder, B. J., Diehn, M., Levy, R., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0280-2 · Cited by 314
Learned protein embeddings for machine learning
Yang, K. K., Wu, Z., Bedbrook, C. N., & Arnold, F. H. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty178 · Cited by 311
Deep learning enables high-quality and high-throughput prediction of enzyme commission numbers
Ryu, J. Y., Kim, H. U., & Lee, S. Y. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821905116 · Cited by 293
Bioinformatics Methods for Mass Spectrometry-Based Proteomics Data Analysis
Chen, C., Hou, J., Tanner, J. J., & Cheng, J. (2020) · International Journal of Molecular Sciences · doi:10.3390/ijms21082873 · Cited by 280
The transformational role of GPU computing and deep learning in drug discovery
Pandey, M., Fernandez, M., Gentile, F., Isayev, O., Tropsha, A., Stern, A. C., & Cherkasov, A. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00463-x · Cited by 255
Deep Learning in Protein Structural Modeling and Design
Gao, W., Mahajan, S. P., Sulam, J., & Gray, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100142 · Cited by 227
Gene2Vec: Distributed Representation of Genes Based on Co-Expression
Du, J., Jia, P., Dai, Y., Tao, C., Zhao, Z., & Zhi, D. (2018) · doi:10.1101/286096 · Cited by 204
Emerging technologies and analytics for a new era of value-centered marketing in healthcare
Agarwal, R., Dugas, M., Gao, G. (., & Kannan, P. K. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00692-4 · Cited by 175
Protein language model embeddings for fast, accurate, alignment-free protein structure prediction
Weissenow, K., Heinzinger, M., & Rost, B. (2021) · doi:10.1101/2021.07.31.454572 · Cited by 139
(2022) · doi:10.1016/j.str.2022.05.001 · Cited by 139
Adly, A. S., Adly, A. S., & Adly, M. S. (2020) · Journal of Medical Internet Research · doi:10.2196/19104 · Cited by 138
Protein embeddings and deep learning predict binding residues for various ligand classes
(2021) · doi:10.1038/s41598-021-03431-4 · Cited by 120
Deep learning in biomedical data science
Baldi, P. (2018) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-080917-013343 · Cited by 119
rawMSA: End-to-end Deep Learning using raw Multiple Sequence Alignments
Mirabello, C., & Wallner, B. (2019) · PLOS ONE · doi:10.1371/journal.pone.0220182 · Cited by 105
Amino acid encoding methods for protein sequences: a comprehensive review and assessment
Jing, X., Dong, Q., Hong, D., & Lu, R. (2020) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2911677 · Cited by 97
DeepCPI: A Deep Learning-Based Framework for Large-Scale in Silico Drug Screening
Wan, F., Zhu, Y., Hu, H., Dai, A., Cai, X., Chen, L., Gong, H., Xia, T., Yang, D., Wang, M.-W., et al. (2019) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2019.04.003 · Cited by 92
Large language models improve annotation of prokaryotic viral proteins
Flamholz, Z. N., Biller, S. J., & Kelly, L. (2024) · Nature Microbiology · doi:10.1038/s41564-023-01584-8 · Cited by 92
Protein design via deep learning
Ding, W., Nakai, K., & Gong, H. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac102 · Cited by 66
Wei, B., Zhang, Y., & Gong, X. (2022) · Scientific Reports · doi:10.1038/s41598-022-23014-1 · Cited by 53
Jin, W., Brannan, K. W., Kapeli, K., Park, S. S., Tan, H. Q., Gosztyla, M. L., Mujumdar, M., Ahdout, J., Henroid, B., Rothamel, K., et al. (2023) · Molecular Cell · doi:10.1016/j.molcel.2023.06.019 · Cited by 52
Cosentino, S., Sriswasdi, S., & Iwasaki, W. (2024) · Genome Biology · doi:10.1186/s13059-024-03298-4 · Cited by 50
An analysis of protein language model embeddings for fold prediction
Villegas-Morcillo, A., Gomez, A. M., & Sanchez, V. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac142 · Cited by 46
GraphCPI: Graph Neural Representation Learning for Compound-Protein Interaction
Quan, Z., Guo, Y., Lin, X., Wang, Z.-J., & Zeng, X. (2019) · 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8983267 · Cited by 35
Seq-InSite: sequence supersedes structure for protein interaction site prediction
Hosseini, S., Golding, G. B., & Ilie, L. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btad738 · Cited by 26
Zhao, M., Yuan, M., Yang, Y., & Xu, S. X. (2023) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2022.3225296 · Cited by 22
Scoring alignments by embedding vector similarity
(2024) · doi:10.1093/bib/bbae178 · Cited by 8
(2025) · doi:10.1016/j.jhazmat.2025.139625 · Cited by 7
Exploring the Knowledge of an Outstanding Protein to Protein Interaction Transformer
Yang, S., Cheng, P., Liu, Y., Feng, D., & Wang, S. (2024) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2024.3381825 · Cited by 6
Protein embeddings and local alignments
Malec, J., Golding, G. B., & Ilie, L. (2026) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.12.002 · Cited by 5
Zheng, S. (2025) · PLOS ONE · doi:10.1371/journal.pone.0313812 · Cited by 4
AlphaMut: A Deep Reinforcement Learning Model to Suggest Helix-Disrupting Mutations
Bhargav, P., & Mukherjee, A. (2025) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.4c01387 · Cited by 2
Metadata
- Authors
- Asgari, E., & Mofrad, M. R. K.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-11-10
- Volume
- 10
- Issue
- 11
- Pages
- e0141287
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W1501531009
- Cited by
- 821
- References
- 39
- FWCI
- 13.25
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.