CoChat
Sign Up
Cited by 206Protein Structure and DynamicsMachine Learning in BioinformaticsRNA and protein synthesis mechanisms

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Transfer learning to leverage larger datasets for improved prediction of protein stability changes

Open access
Dieckhaus, Henry, Brocidiacono, Michael, Randolph, Nicholas Z., Kuhlman, Brian·2024·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.2314853121PMC PMC10861915

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Amino acid mutations that lower a protein’s thermodynamic stability are implicated in numerous diseases, and engineered proteins with enhanced stability can be important in research and medicine. Computational methods for predicting how mutations perturb protein stability are, therefore, of great interest. Despite recent advancements in protein design using deep learning, in silico prediction of stability changes has remained challenging, in part due to a lack of large, high-quality training datasets for model development. Here, we describe ThermoMPNN, a deep neural network trained to predict stability changes for protein point mutations given an initial structure. In doing so, we demonstrate the utility of a recently released megascale stability dataset for training a robust stability model. We also employ transfer learning to leverage a second, larger dataset by using learned features extracted from ProteinMPNN, a deep neural network trained to predict a protein’s amino acid sequence given its three-dimensional structure. We show that our method achieves state-of-the-art performance on established benchmark datasets using a lightweight model architecture that allows for rapid, scalable predictions. Finally, we make ThermoMPNN readily available as a tool for stability prediction and design.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.2314853121

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. ThermoMPNN architecture and primary dataset statistics. (A) Model architecture of ThermoMPNN, a graph neural network trained on embeddings extracted from a pretrained sequence recovery model (ProteinMPNN, Left panel) to predict thermostability changes caused by protein point mutations. The input protein is passed through ProteinMPNN, where the learned embeddings from each decoder layer are extracted and concatenated with the learned sequence embedding to create a vector representation of the residue environment. This vector is passed through a light attention block (LA, purple block) which uses self-attention to reweight the vector based on learned context. Finally, a multilayer perceptron (MLP, red block) predicts a ΔΔG° by subtracting the predicted ΔG° for the wild-type amino acid from the predicted ΔG° of the mutant amino acid. (B) Curation, clustering, and data splitting procedure for the Megascale and Fireprot datasets used in this study. Each split is labeled with its total number of mutations, and homologues are shown in yellow. Each clustering result is labeled with the number of clusters in each dataset. (C) Distribution of mutations per protein for each dataset. (D) Distribution of protein length for each dataset. (E) Percentage of mutations to alanine compared to other polar and nonpolar residues for each dataset, along with natural residue abundance for all proteins in the SwissProt database for comparison.
Fig. 1. ThermoMPNN architecture and primary dataset statistics. (A) Model architecture of ThermoMPNN, a graph neural network trained on embeddings extracted from a pretrained sequence recovery model (ProteinMPNN, Left panel) to predict thermostability changes caused by protein point mutations. The input protein is passed through ProteinMPNN, where the learned embeddings from each decoder layer are extracted and concatenated with the learned sequence embedding to create a vector representation of the residue environment. This vector is passed through a light attention block (LA, purple block) which uses self-attention to reweight the vector based on learned context. Finally, a multilayer perceptron (MLP, red block) predicts a ΔΔG° by subtracting the predicted ΔG° for the wild-type amino acid from the predicted ΔG° of the mutant amino acid. (B) Curation, clustering, and data splitting procedure for the Megascale and Fireprot datasets used in this study. Each split is labeled with its total number of mutations, and homologues are shown in yellow. Each clustering result is labeled with the number of clusters in each dataset. (C) Distribution of mutations per protein for each dataset. (D) Distribution of protein length for each dataset. (E) Percentage of mutations to alanine compared to other polar and nonpolar residues for each dataset, along with natural residue abundance for all proteins in the SwissProt database for comparison.CC BY-NC-ND · PubMed Central
Fig. 2. ThermoMPNN performance analysis. (A) Spearman correlation coefficient (SCC) and PCC calculated on predictions for models trained (in triplicate) using different datasets on Megascale (blue circle) and Fireprot (red diamond) test datasets. Models were also evaluated after either concurrent (Both conc) or sequential (Both seq) cotraining, as well as training on only a subset of Megascale data including a similar number of proteins (Megascale SP) or a similar number of mutations (Megascale SM) to the Fireprot data. (B) ThermoMPNN predictions (ΔΔG°pred) on the Megascale test (Left, N = 28,312) and Fireprot HF (Right, N = 2,578) datasets plotted vs. their respective experimental values (ΔΔG°exp) with identity line (dashed gray line) for reference. Points are colored by local density of mutations (lighter = denser) using a fitted Gaussian kernel density estimate (KDE) function. (C) ThermoMPNN RMSE for all Megascale and Fireprot (HF) proteins (minimum 25 mutations, N = 320 proteins) plotted by length, with joint kernel density estimate (KDE) plots of length (Top) and RMSE (Right) distributions of natural (N = 211) and de novo (N = 109) proteins. (D) ThermoMPNN PCC on experimental and computationally modeled (AlphaFold) structures for all Megascale and Fireprot (HF) proteins solved by X-ray crystallography (Left, N = 113 proteins) and NMR (Right, N = 152 proteins).
Fig. 2. ThermoMPNN performance analysis. (A) Spearman correlation coefficient (SCC) and PCC calculated on predictions for models trained (in triplicate) using different datasets on Megascale (blue circle) and Fireprot (red diamond) test datasets. Models were also evaluated after either concurrent (Both conc) or sequential (Both seq) cotraining, as well as training on only a subset of Megascale data including a similar number of proteins (Megascale SP) or a similar number of mutations (Megascale SM) to the Fireprot data. (B) ThermoMPNN predictions (ΔΔG°pred) on the Megascale test (Left, N = 28,312) and Fireprot HF (Right, N = 2,578) datasets plotted vs. their respective experimental values (ΔΔG°exp) with identity line (dashed gray line) for reference. Points are colored by local density of mutations (lighter = denser) using a fitted Gaussian kernel density estimate (KDE) function. (C) ThermoMPNN RMSE for all Megascale and Fireprot (HF) proteins (minimum 25 mutations, N = 320 proteins) plotted by length, with joint kernel density estimate (KDE) plots of length (Top) and RMSE (Right) distributions of natural (N = 211) and de novo (N = 109) proteins. (D) ThermoMPNN PCC on experimental and computationally modeled (AlphaFold) structures for all Megascale and Fireprot (HF) proteins solved by X-ray crystallography (Left, N = 113 proteins) and NMR (Right, N = 152 proteins).CC BY-NC-ND · PubMed Central
Fig. 3. Literature comparison with ThermoMPNN predictions on selected case study proteins. At middle, bar plot of PCC for ThermoMPNN and selected literature methods. Methods trained/parameterized without homologous proteins (“unbiased training”) are indicated with dark bars, while those with data leakage concerns are indicated with light bars. RaSP and PROSTATA models were retrained on the Megascale dataset for this comparison. At Left and Right, scatter plots of stability change predictions (ΔΔG°pred) vs. experimental measurements (ΔΔG°exp) for external test proteins p53 (PDB ID: 2OCJ, N = 42 mutations), staphylococcus nuclease (1EY0, N = 559), myoglobin (1BZ6, N = 134), and T4 lysozyme (2LZM, N = 309) with fitted regression line (shaded region = 95% CI). The cartoon representation of each protein is included for reference.
Fig. 3. Literature comparison with ThermoMPNN predictions on selected case study proteins. At middle, bar plot of PCC for ThermoMPNN and selected literature methods. Methods trained/parameterized without homologous proteins (“unbiased training”) are indicated with dark bars, while those with data leakage concerns are indicated with light bars. RaSP and PROSTATA models were retrained on the Megascale dataset for this comparison. At Left and Right, scatter plots of stability change predictions (ΔΔG°pred) vs. experimental measurements (ΔΔG°exp) for external test proteins p53 (PDB ID: 2OCJ, N = 42 mutations), staphylococcus nuclease (1EY0, N = 559), myoglobin (1BZ6, N = 134), and T4 lysozyme (2LZM, N = 309) with fitted regression line (shaded region = 95% CI). The cartoon representation of each protein is included for reference.CC BY-NC-ND · PubMed Central
Fig. 4. Comparison of ThermoMPNN and ProteinMPNN amino acid preferences. Change in amino acid preferences of ThermoMPNN compared to ProteinMPNN if the most favorable mutation is chosen at every position for all Megascale and Fireprot (HF) proteins. Cysteine mutations were excluded due to potential disulfide formation (SI Appendix, Fig. S5).
Fig. 4. Comparison of ThermoMPNN and ProteinMPNN amino acid preferences. Change in amino acid preferences of ThermoMPNN compared to ProteinMPNN if the most favorable mutation is chosen at every position for all Megascale and Fireprot (HF) proteins. Cysteine mutations were excluded due to potential disulfide formation (SI Appendix, Fig. S5).CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

Sorted by citations

See all 206

Related

See all

Metadata

Authors
Dieckhaus, H., Brocidiacono, M., Randolph, N. Z., & Kuhlman, B.
Affiliations
University of North Carolina at Chapel Hill (US)
Corresponding author
Kuhlman, Brian
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2024-01-29
Volume
121
Issue
6
Pages
e2314853121-e2314853121
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.2314853121
PMC
PMC10861915
OpenAlex
W4391308485
Cited by
205
References
47
Citations by year
2026: 104 · 2025: 74 · 2024: 25
FWCI
27.63
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by-nd)
Funders
  • HHS | NIH | National Institute of General Medical Sciences (R35GM131923)
  • NSF | EDU | Division of Graduate Education (DGE-2040435)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
Keywords
protein stability prediction, transfer learning, ProteinMPNN, point mutations, protein thermodynamic stability, deep neural network, protein design, learned features, engineered proteins

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review