CoChat
Sign Up
Cited by 486Machine Learning in BioinformaticsProtein Structure and DynamicsFractal and DNA sequence analysis

Learning the protein language: Evolution, structure, and function

Open access
Bepler, Tristan, Berger, Bonnie·2021·Cell Systems
DOI 10.1016/j.cels.2021.05.017PMC PMC8238390
Open this paper in CoChat — free accountOpen PDF

Abstract

Language models have recently emerged as a powerful machine-learning approach for distilling information from massive protein sequence databases. From readily available sequence data alone, these models discover evolutionary, structural, and functional organization across protein space. Using language models, we can encode amino-acid sequences into distributed vector representations that capture their structural and functional properties, as well as evaluate the evolutionary fitness of sequence variants. We discuss recent advances in protein language modeling and their applications to downstream protein property prediction problems. We then consider how these models can be enriched with prior biological knowledge and introduce an approach for encoding protein structural knowledge into the learned representations. The knowledge distilled by these models allows us to improve downstream function prediction through transfer learning. Deep protein language models are revolutionizing protein biology. They suggest new ways to approach protein and therapeutic design. However, further developments are needed to encode strong biological priors into protein language models and to increase their accessibility to the broader community.

AI snapshot

Written from the paper’s full text.

TL;DR

Protein language models learn evolutionary, structural, and functional organization from large sequence databases, and adding structure supervision improves downstream prediction through transfer learning. The authors present a multi-task LSTM trained on 76 million sequences plus 28,000 structures and show gains in structural similarity, transmembrane prediction, and sequence-to-phenotype prediction.

Key findings

  • A multi-task LSTM (MT-LSTM) trained with masked language modeling, residue-residue contact prediction, and SCOP structural similarity prediction learned embeddings that “dramatically outperform” sequence comparison methods and even outperform TMalign for structural similarity; structure supervision was critical for fine-grained organization, and MT-LSTM outperformed the prior SSA-LSTM.
  • For transmembrane region prediction, BiLSTM+CRF models using MT-LSTM embeddings significantly outperformed DLM-LSTM embeddings (paired t-test, p=0.044) and also outperformed existing predictors and SSA-LSTM.
  • For sequence-to-phenotype prediction on deep mutational scanning datasets, MT-LSTM embeddings with Gaussian process regression achieved an average Spearman correlation of 0.65, significantly outperforming ECNet’s 0.60 (paired t-test, p=0.006).
  • The paper notes that language models predicted SARS-CoV-2 spike mutations with high antibody-escape potential, but as of publication viral escape had not yet been experimentally verified.

Methods

  • The authors trained a three-layered bidirectional LSTM with skip connections and 1024 hidden units in each direction per layer. The MT-LSTM combined three losses: masked language modeling (10% of amino acids randomly masked), residue-residue contact prediction (contacts defined as residues within 8 Å), and SCOP structural similarity prediction via ordinal regression; MT-LSTM loss weights were λmasked=0.5, λcontact=0.9, λsimilarity=0.1, while DLM-LSTM used only λmasked=1.
  • Training data were 76,215,872 UniRef90 sequences (July 2018) and 28,010 SCOPe ASTRAL 2.06 protein sequences with structures, split into 22,408 training and 5,602 testing sequences. Models were trained for 1,000,000 parameter updates with Adam at learning rate 0.0001 on a single NVIDIA V100 GPU; training took roughly 13 days for DLM-LSTM and 51 days for MT-LSTM.
  • Evaluations included structural similarity on 100,000 randomly sampled SCOPe ASTRAL test pairs, transmembrane prediction on the TOPCONS2 dataset of 5,154 proteins using 10-fold cross validation, and sequence-to-phenotype prediction on deep mutational scanning datasets using 5-fold cross validation and Gaussian process regression with an RBF kernel.

Limitations

  • The paper states that further developments are needed to encode strong biological priors into protein language models and to increase their accessibility to the broader community; it also states that current models make heavily simplified phylogenetic assumptions, treating sequences as independent draws from a star phylogeny.
  • The paper states practical limitations including high compute costs, murky design guidelines, ad hoc model development, quadratic scaling of transformers with sequence length (requiring long sequences to be excluded or truncated), and poor learning of long-range dependencies by RNNs.
  • Evident from the text: the new results come from a single MT-LSTM architecture and specific benchmarks, and the SARS-CoV-2 escape predictions had not been experimentally verified as of publication.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.cels.2021.05.017

Cites

130 of 130 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 53 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Increasing model size improves transfer learning performance.

    “We also find that increasing model size improves transfer learning performance.”
    Quote in the paperSubstantial claim
  • Findingown data

    Including structure supervision in language model training improves the organization of protein sequences by structure in embedding space compared to DLM-LSTM.

    “However, this organization is improved when we include structure supervision in the language model training (Figure 4b).”
    Quote in the paperSubstantial claim
  • Findingown data

    MT-LSTM representations significantly outperform DLM-LSTM embeddings on transmembrane prediction (paired t-test, p=0.044).

    “Furthermore, representations learned by our MT-LSTM model significantly outperform (paired t-test, p=0.044) the embeddings learned by our DLM-LSTM model on this application (Figure 5b).”
    Quote in the paperSubstantial claim
  • Findingcitation

    The multi-task learning approach outperforms a two-step learning approach (SSA-LSTM) for structural similarity prediction.

    “Furthermore, the multi-task learning approach outperforms a two-step learning approach presented previously (SSA-LSTM)27.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] Bepler T & Berger B Learning protein sequence embeddings using information from structure. in International Conference on Learning Representations (2019). PMC12947628 41768824 source

      Not checked in the free run.

  • Findingown data

    By comparing vector embeddings in the learned semantic representation space, we can find structurally related proteins even though their sequences are not closely related.

    “Because we embed sequences into a semantic representation space, we can find structurally related proteins even though their sequences are not closely related (Figure 4c, Supplemental Table 1).”
    Quote in the paperSubstantial claim
  • May combine two claims. 13
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 8

  • May combine two claims.

    Findingown data

    Learned semantic embeddings from MT-LSTM dramatically outperform sequence comparison methods and even outperform structure comparison with TMalign when predicting structural similarity.

    “We find that our learned semantic embeddings dramatically outperform the sequence comparison methods and even outperform structure comparison with TMalign when predicting structural similarity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Training protein language models with self-supervision on large natural sequence data and with structure supervision on a smaller set of sequence-structure pairs enriches learned representations and improves downstream prediction problems through transfer learning.

    “The knowledge distilled by these models allows us to improve downstream function prediction through transfer learning.”
    • Also stated in: § Multi-task language models capture the semantic organization of proteins · boosted
      “Here, we will demonstrate that training protein language models with self-supervision on a large amount of natural sequence data and with structure supervision on a smaller set of sequence, structure pairs enriches the learned representations and translates into improvements in downstream prediction problems (Figure 3).”
    • Also stated in: § Transfer learning improves downstream applications · plain
      “Learned protein representations improve predictive ability for downstream prediction problems through transfer learning (Figure 5a).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Many problems in biology are small data problems; quantitative protein characterization assays are rarely high throughput and methods are needed that can generalize given only 10s to 100s of experimental measurements.
    • Rests on an assumption: We are often interested in extrapolating from data collected over a small region of protein sequence space to other sequences, often with little to no homology.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    MT-LSTM embedding powered GP predicts phenotypes of unobserved sequence variants across datasets better than existing methods, achieving average Spearman correlation 0.65 vs ECNet 0.60 (paired t-test, p=0.006).

    “We find that we can predict the phenotypes of unobserved sequence variants across datasets better than existing methods (Figure 5c).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Many problems in biology are small data problems; quantitative protein characterization assays are rarely high throughput and methods are needed that can generalize given only 10s to 100s of experimental measurements.
    • Rests on an assumption: We are often interested in extrapolating from data collected over a small region of protein sequence space to other sequences, often with little to no homology.

  • May combine two claims.

    Findingown data

    BiLSTM+CRFs with DLM-LSTM and MT-LSTM embeddings outperform existing transmembrane predictors and a BiLSTM+CRF using SSA-LSTM.

    “We find that the BiLSTM+CRFs with our new embeddings (DLM-LSTM and MT-LSTM) outperform existing transmembrane predictors and a BiLSTM+CRF using our previous smaller embedding model (SSA-LSTM).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Structural supervision is critical for learning well-organized embeddings at a fine-grained level, because DLM-LSTM representations alone do not perform well at this task.

    “Interestingly, we observe that the structural supervision component is critical for learning well organized embeddings at a fine-grained level, because the DLM-LSTM representations alone do not perform well at this task (Supplemental Table 1).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In NLP, the number of parameters, rather than specific architectural choices, is the best indicator of model performance.

    “This parallels the general trend in natural language processing, where the number of parameters, rather than specific architectural choices, is the best indicator of model performance 52.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Kaplan J Scaling Laws for Neural Language Models. arXiv [cs.LG] (2020).

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Structure is more informative of function than sequence and structure is encoded by sequence.

    “Although structure is more informative of function than sequence 57,58 and structure is encoded by sequence, predicting structure remains hard, particularly due to the relative paucity of structural relative to sequence data.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [57] Zhang C & Kim S-H Overview of structural genomics: from structure to function. Curr. Opin. Chem. Biol 7, 28– 32 (2003). 12547423 source

      Not checked in the free run.

    • cites [58] Shin DH Structure-based inference of molecular functions of proteins of unknown function from Berkeley Structural Genomics Center. J. Struct. Funct. Genomics 8, 99– 105 (2007). 17764033 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Language models learn summary representations that can be used to capture sequence relationships and link sequence to function via transfer learning.

    “They also learn summary representations, powerful features that can be used to better capture sequence relationships and link sequence to function via transfer learning 27,28,32-35.”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [27] Bepler T & Berger B Learning protein sequence embeddings using information from structure. in International Conference on Learning Representations (2019). PMC12947628 41768824 source

      Not checked in the free run.

    • cites [28] Alley EC, Khimulya G, Biswas S, AlQuraishi M & Church GM Unified rational protein engineering with sequence-based deep representation learning. Nat. Methods 16, 1315– 1322 (2019). 31636460 PMC7067682 source

      Not checked in the free run.

    • cites [32] Rives A Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. bioRxiv (2019). PMC8053943 33876751 source

      Not checked in the free run.

    • cites [33] Rao R Evaluating Protein Transfer Learning with TAPE. Adv. Neural Inf. Process. Syst 32, 9689– 9701 (2019). 33390682 PMC7774645 source

      Not checked in the free run.

    • cites [34] Luo Y Evolutionary Context-Integrated Deep Sequence Modeling for Protein Engineering. in Research in Computational Molecular Biology 261– 263 (Springer International Publishing, 2020).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [35] Hie B, Bryson BD & Berger B Leveraging Uncertainty in Machine Learning Accelerates Biological Discovery and Design. Cell Syst 11, 461– 477.e9 (2020). 33065027 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    There are examples of highly similar protein sequences having radically different structures and functions and of sequences that can form multiple folds.

    “Although it is generally believed that proteins with similar sequences form similar structures, there are also interesting examples of highly similar protein sequences having radically different structures and functions 60,61 and of sequences that can form multiple folds 62.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [60] Kosloff M & Kolodny R Sequence-similar, structure-dissimilar protein pairs in the PDB. Proteins 71, 891– 902 (2008). 18004789 PMC2673347 source

      Not checked in the free run.

    • cites [61] Wei KY Computational design of closely related proteins that adopt two well-defined but structurally divergent folds. Proc. Natl. Acad. Sci. U. S. A 117, 7208– 7215 (2020). 32188784 PMC7132107 source

      Not checked in the free run.

    • cites [62] James LC & Tawfik DS Conformational diversity and protein evolution--a 60-year-old hypothesis revisited. Trends Biochem. Sci 28, 361– 368 (2003). 12878003 source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning

    Unrelated protein sequences can form the same structural fold and are semantically similar, but this relationship cannot be deduced from sequences alone; supervision is required to learn that these sequences belong to the same semantic category.

    “For example, unrelated protein sequences can form the same structural fold and, therefore, are semantically similar. However, we cannot deduce this relationship from sequences alone. Supervision is required to learn that these sequences belong to the same semantic category.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    There are aspects of protein biology that may not be discoverable by statistical sequence models alone; supervision representing known protein structure, function, and other prior knowledge may be necessary to encode distant sequence relationships into learned embeddings.

    “These phenomena suggest that there are aspects of protein biology that may not be discoverable by statistical sequence models alone. Supervision that represents known protein structure, function, and other prior knowledge may be necessary to encode distant sequence relationships into learned embeddings.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Proteins with similar structures and functions still exist below 30% sequence identity, but they can no longer be detected from sequence similarity alone and it is unclear whether their functions are conserved.
    • Rests on an assumption: Evolutionary innovation requires that protein function can change with only a few mutations.

  • May combine two claims.

    Premisecitation

    Language models learn the probability of a sequence occurring and this can be directly applied to predict the fitness of sequence mutations.

    “Language models learn the probability of a sequence occurring and this can be directly applied to predict the fitness of sequence mutations 29-31 .”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [29] Riesselman AJ, Ingraham JB & Marks DS Deep generative models of genetic variation capture the effects of mutations. Nat. Methods 15, 816– 822 (2018). 30250057 PMC6693876 source

      Not checked in the free run.

    • cites [30] Hie B, Zhong E, Bryson B & Berger B Learning mutational semantics. Adv. Neural Inf. Process. Syst 33, (2020).

      Not checked in the free run.

    • cites [31] Hie B, Zhong ED, Berger B & Bryson B Learning the language of viral evolution and escape. Science 371, 284– 288 (2021). 33446556 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Language models capture complex relationships between residues in protein sequences by condensing information from enormous protein sequence databases.

    “Language models capture complex relationships between residues in protein sequences by condensing information from enormous protein sequence databases.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Fine-tuning pre-trained models can help with model selection but does not fully resolve it.

    “Fine-tuning pre-trained models can help with this problem but does not fully resolve it.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: For individual protein families or application-specific protein datasets, brute force hyperparameter search is the gold standard for selecting model architectures and number of parameters.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    High compute costs and murky design guidelines make developing new protein language models an expensive, time-consuming, and ad hoc process.

    “High compute costs and murky design guidelines mean that developing new models is often an expensive, time consuming, and ad hoc process.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The study used 5-fold cross validation on each deep mutational scanning dataset to compare methods.

    “To compare methods, we follow Luo et al and perform 5-fold cross validation on each deep mutational scanning dataset 34”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Luo et al

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [34] Luo Y Evolutionary Context-Integrated Deep Sequence Modeling for Protein Engineering. in Research in Computational Molecular Biology 261– 263 (Springer International Publishing, 2020).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study used 10-fold cross validation to compare methods for transmembrane prediction.

    “We compare methods by 10-fold cross validation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    New linear complexity attention mechanisms may help alleviate the sequence length limitation.

    “New linear complexity attention mechanisms may help to alleviate this limitation 95,96.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [95] Choromanski KM Rethinking Attention with Performers. in International Conference on Learning Representations (2020).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [96] Wang S, Li BZ, Khabsa M, Fang H & Ma H Linformer: Self-Attention with Linear Complexity. (2020). source

      Not checked in the free run.

    • Rests on an assumption: Transformers scale quadratically with sequence length.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Luo et al. presented a method for combining language model-based representations with local evolutionary context-based representations (ECNet) and demonstrated that these representations were powerful for sequence-to-phenotype mapping on a panel of deep mutational scanning datasets.

    “Recently, Luo et al. presented a method for combining language model-based representations with local evolutionary context-based representations (ECNet) and demonstrated that these representations were powerful for sequence-to-phenotype mapping on a panel of deep mutational scanning datasets 34.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Luo Y Evolutionary Context-Integrated Deep Sequence Modeling for Protein Engineering. in Research in Computational Molecular Biology 261– 263 (Springer International Publishing, 2020).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

19 of 19

  • The paper relies on this without support.

    Many problems in biology are small data problems; quantitative protein characterization assays are rarely high throughput and methods are needed that can generalize given only 10s to 100s of experimental measurements.

    “A key challenge in biology is that many problems are small data problems. Quantitative protein characterization assays are rarely high throughput and methods are needed that can generalize given only 10s to 100s of experimental measurements.”

  • The paper relies on this without support.

    We are often interested in extrapolating from data collected over a small region of protein sequence space to other sequences, often with little to no homology.

    “Furthermore, we are often interested in extrapolating from data collected over a small region of protein sequence space to other sequences, often with little to no homology.”

  • The paper relies on this without support.

    Proteins with similar structures and functions still exist below 30% sequence identity, but they can no longer be detected from sequence similarity alone and it is unclear whether their functions are conserved.

    “Often called the “twilight zone” of protein sequence homology, proteins with similar structures and functions still exist below this level, but they can no longer be detected from sequence similarity alone and it is unclear whether their functions are conserved.”

  • The paper relies on this without support.

    Evolutionary innovation requires that protein function can change with only a few mutations.

    “Evolutionary innovation requires that protein function can change with only a few mutations.”

  • The paper relies on this without support.

    Proteins exist in physical space.

    “Proteins are objects that exist in physical space.”

  • The paper relies on this without support.

    Many fundamental evolutionary processes that give rise to the diversity of protein sequences observed today are understood.

    “Similarly, we understand many of the fundamental evolutionary processes that give rise to the diversity of protein sequences observed today.”

  • The paper relies on this without support.

    Physics and evolution are key properties of proteins.

    “These two elements, physics and evolution, are the key properties of proteins and our models might benefit from being structured explicitly to incorporate evolutionary and physics-based inductive biases.”

  • The paper relies on this without support.

    Biological sequences are not natural language.

    “Biological sequences are not natural language”

  • The paper relies on this without support.

    Filtering sequences based on percent identity is a conventional crude address to the phylogenetic assumption problem.

    “Conventionally, this problem is crudely addressed by filtering sequences based on percent identity.”

  • The paper relies on this without support.

    At the upper limit of large natural protein databases, larger models continue to yield improved performance.

    “At the upper limit of large natural protein databases, larger models continue to yield improved performance.”

  • The paper relies on this without support.

    For individual protein families or application-specific protein datasets, brute force hyperparameter search is the gold standard for selecting model architectures and number of parameters.

    “However, for individual protein families or other application specific protein data sets, the gold standard is to select model architectures and number of parameters via brute force hyperparameter search methods.”

  • The paper relies on this without support.

    Transformers scale quadratically with sequence length.

    “Transformers scale quadratically with sequence length”

  • The paper relies on this without support.

    Recurrent neural networks scale linearly with sequence length.

    “This problem is less extreme for recurrent neural networks, which scale linearly with sequence length”

  • The paper relies on this without support.

    Increasing model size, compute power, and dataset size will continue to improve performance of protein language models.

    “Increasing model size, compute power, and dataset size will only continue to improve performance of protein language models.”

  • The paper relies on this without support.

    Protein language models are transforming computational protein biology today due to their ease of use and widespread applicability.

    “Already, these methods are transforming computational protein biology today due to their ease of use and widespread applicability.”

  • The paper relies on this without support.

    Augmenting language models with protein-specific properties such as structure and function offers an already successful route towards richer representations and novel biology.

    “Furthermore, augmenting language models with protein specific properties such as structure and function offers one already successful route towards even richer representations and novel biology.”
16 of 16

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Gated Attention Unit (GAU) has shown great promise as an alternative to multi-headed self-attention with smaller memory consumption and faster convergence rate.

    “Instead of using multi-headed self-attention (MHSA), we adopt the Gated Attention Unit (GAU) (line 8 in Algorithm 1), which has shown great promise as an alternative to MHSAs with smaller memory consumption and faster convergence rate (25).”

    From High-resolution <i>de novo</i> structure prediction from primary sequence (2022) · cites this paper as [25]

  • The original GAU suggests that relu^2 performs better compared to softmax in terms of both computation speed and convergence rate.

    “The original GAU in (25) also suggests that relu 2 (•) performs better compared to softmax in terms of both computation speed and convergence rate.”

    From High-resolution <i>de novo</i> structure prediction from primary sequence (2022) · cites this paper as [25]

Cites

Flagged first, then by citations

3 of the 133 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1
  2. Remmert, 2011, HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment, Nat. Methods, 9, 173, 10.1038/nmeth.1818 · DOI · Paper page
  3. Remmert, 2011, HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment, Nat. Methods, 9, 173, 10.1038/nmeth.1818 · DOI · Paper page
  4. Tsirigos, 2015, The TOPCONS web server for consensus prediction of membrane protein topology and signal peptides, Nucleic Acids Res., 43, W401-7, 10.1093/nar/gkv485 · DOI

Cited by

Sorted by citations

Related

Authors
Bepler, T., & Berger, B.
Affiliations
  • New York Structural Biology Center (US)
  • MIT Computer Science and Artificial Intelligence Laboratory (US)
Corresponding author
  • Bepler, Tristan
  • Berger, Bonnie
ORCID
Venue
Cell Systems
Publisher
Elsevier BV
Published
2021-06
Volume
12
Issue
6
Pages
654-669.e3
ISSN
2405-4712 (ISSN-L)
DOI
10.1016/j.cels.2021.05.017
PMC
PMC8238390
OpenAlex
W3166142427
Cited by
483
References
133
Citations by year
2026: 87 · 2025: 120 · 2024: 127 · 2023: 82 · 2022: 55 · 2021: 12
FWCI
21.74
Type
journal article
Language
en
Open access
Licence
Locations
  • Cell Systems (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
Funders
  • NIH (R35 GM141861)
  • Simons Foundation (SF349247)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Fractal and DNA sequence analysis
Keywords
protein language models, protein sequence representation, protein structure prediction, protein function prediction, knowledge distillation, transfer learning, protein design, therapeutic design
MeSH terms
Machine Learning, Amino Acid Sequence, Language, Proteins, Databases, Protein

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review