CoChat
Sign Up
Cited by 33Protein Structure and DynamicsMachine Learning in BioinformaticsGenomics and Rare Diseases

On the biases in predictions of protein stability changes upon variations: the INPS test case

Montanucci, Ludovica, Savojardo, Castrense, Martelli, Pier Luigi, Casadio, Rita, Fariselli, Piero·2019·Bioinformatics
DOI 10.1093/bioinformatics/bty979
Open this paper in CoChat — free account

Abstract

INPS (Fariselli et al., 2015) is a simple method to predict the protein stability changes upon single point variation. It is based on Support Vector Machine and it was trained using 7 input features that are: (i) Blosum62 substitution, (ii) residue mutability, (iii) molecular weight wild type, (iv) molecular weight variant, (v) hydrophobicity wild type, (vi) hydrophobicity variant, (vii) difference between Viterbi scores of the wild type and variant (Fariselli et al., 2015). The first two input features are not anti-symmetric, the last is anti-symmetric by construction, while the other features can be learned to be anti-symmetric. To enforce anti-symmetry INPS was trained using both direct and inverse variations (Fariselli et al., 2015). When the alignment is poor we can expect a deviation from the anti-symmetry. INPS3D (Savojardo et al., 2016), adds to INPS information about contact potential and solvent accessibility. The solvent accessibility is intrinsically non anti-symmetric, and for this reason INPS3D was trained only with direct variations (Savojardo et al., 2016). The Ssym dataset is a manually curated selection of variations from the ProTherm database (Pucci et al., 2018). It contains variations with experimental ΔΔG values for which the 3D structures of both the wild-type and variant proteins were solved by X-ray crystallography. Ssym consists of 684 variations, half of which are direct (reported in the literature) and half are obtained by anti-symmetry (Pucci et al., 2018). To overcome the bias problem, a new method, PopMuSiCSYM, was explicitly developed to furnish completely anti-symmetric predictions of the variations (Pucci et al., 2015, 2018). The authors of the Ssym dataset estimated the performance of fifteen selected predictors using the average bias parameter (<δ> = ∑(ΔΔGdir+ΔΔGinv⁠)/N) and the linear correlation coefficient (rdir-inv) between the predicted ΔΔG values of the direct and the corresponding inverse variations. A perfect predictor should have bias close to zero (<δ> = 0) and a correlation coefficient close to -1 (rdir-inv = −1). In the same paper, they also tested the performances of these fifteen predictors using the experimental ΔΔG values on both direct and inverse variations using the root mean square deviation (σdir, σinv) and the linear correlation coefficient between the predicted and experimental values (rdir, rinv). In Table 1 we report the predictor performances and biases taken from the original paper (Pucci et al., 2018) and we add those computed using INPS and INPS3D. From Table 1, it is clear that the partial anti-symmetry of the INPS input and its training performed on both direct and inverse variations paid off. INPS shows a very low bias (<δ>, just second best), the highest correlation between direct and inverse variations (rdir-inv) and the highest performance on the inverse variation sets (σinv and rinv). The graph of the correlation between direct and inverse variation predicted by INPS is presented in Figure 1. INPS and INPS3D prediction on Ssym. The INPS (black) and INPS3D (grey) predictions for the 342 pairs (direct and inverse) of single-point variations in Ssym (Pucci et al., 2018) are shown. ΔΔG for the direct (x-axis) and inverse (y-axis) variations are reported as dots in the graph. The ‘ideal’ relationship ΔΔGAB + ΔΔGBA = 0 is shown as a solid line Bias analysis on the Ssym dataset (Pucci et al. 2018) Notes: Standard deviations σ and the average bias <δ> are in kcal/mol. All the values, except those of INPS and INPS3D, are taken from (Pucci et al., 2018). Bold character highlights INPS method. Bias analysis on the Ssym dataset (Pucci et al. 2018) Notes: Standard deviations σ and the average bias <δ> are in kcal/mol. All the values, except those of INPS and INPS3D, are taken from (Pucci et al., 2018). Bold character highlights INPS method. The INPS training set includes some of the proteins in the Ssym dataset (which is also true for all the other tested methods reported in Table 1). For sake of comparison, we also computed the performances of a ‘blind’ INPS. In this case, each ΔΔG prediction is obtained by using a svm model that does not include the protein to predict or a similar one (sequence identity <25%) in the training set. The standard deviations and Pearson correlation coefficients computed for the blind-INPS are: 1.44 kcal/mol and 0.48 for direct variations and 1.45 kcal/mol and 0.47 for inverse variations. The correlation between the predictions of direct and inverse variations is -0.99 while the average bias (measured through the δ) is -0.06 kcal/mol. The structure-based INPS3D has the second best correlation between direct and inverse correlation but a considerably higher bias. This is expected given that INPS3D has not been trained on inverse variations and conains as input feature the solvent accessibility, which is not anti-symmetric. Finally, it must be stressed that the correlations reported in Table 1 are only intended to assess

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/bty979

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Montanucci, L., Savojardo, C., Martelli, P. L., Casadio, R., & Fariselli, P.
Affiliations
  • University of Padua (IT)
  • University of Bologna (IT)
  • University of Turin (IT)
Corresponding author
Fariselli, Piero
ORCID
Venue
Bioinformatics
Publisher
Oxford University Press (OUP)
Published
2018-11-29
Volume
35
Issue
14
Pages
2525-2527
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
DOI
10.1093/bioinformatics/bty979
OpenAlex
W2902559445
Cited by
33
References
9
Citations by year
2026: 2 · 2025: 1 · 2023: 4 · 2022: 6 · 2021: 11 · 2020: 7 · 2019: 2
Type
journal article
Language
en
Open access
closed
Licence
academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
Locations
  • Bioinformatics (publishedVersion)
  • PubMed (publishedVersion)
  • Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
  • Institutional Research Information System University of Turin (University of Turin) (submittedVersion)
  • Padua Research Archive (University of Padova) (submittedVersion)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Genomics and Rare Diseases
Keywords
protein stability prediction, ice nucleating particles, antisymmetry, prediction bias, point mutations, ΔΔG, support vector machine, solvent accessibility, contact potential difference, mean absolute error
MeSH terms
Mutation, Bias, Protein Stability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review