CoChat
Sign Up
Cited by 82Protein Structure and DynamicsMachine Learning in BioinformaticsComputational Drug Discovery Methods

iSEE: Interface structure, evolution, and energy-based machine learning predictor of binding affinity changes upon mutations

Open access
Geng, Cunliang, Vangone, Anna, Folkers, Gert E., Xue, Li C., Bonvin, Alexandre M. J. J.·2019·Proteins: Structure, Function, and Bioinformatics
DOI 10.1002/prot.25630PMC PMC6587874

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Quantitative evaluation of binding affinity changes upon mutations is crucial for protein engineering and drug design. Machine learning‐based methods are gaining increasing momentum in this field. Due to the limited number of experimental data, using a small number of sensitive predictive features is vital to the generalization and robustness of such machine learning methods. Here we introduce a fast and reliable predictor of binding affinity changes upon single point mutation, based on a random forest approach. Our method, iSEE, uses a limited number of i nterface S tructure, E volution, and E nergy‐based features for the prediction. iSEE achieves, using only 31 features, a high prediction performance with a Pearson correlation coefficient (PCC) of 0.80 and a root mean square error of 1.41 kcal/mol on a diverse training dataset consisting of 1102 mutations in 57 protein‐protein complexes. It competes with existing state‐of‐the‐art methods on two blind test datasets. Predictions for a new dataset of 487 mutations in 56 protein complexes from the recently published SKEMPI 2.0 database reveals that none of the current methods perform well (PCC < 0.42), although their combination does improve the predictions. Feature analysis for iSEE underlines the significance of evolutionary conservations for quantitative prediction of mutation effects. As an application example, we perform a full mutation scanning of the interface residues in the MDM2–p53 complex.

AI snapshot

Written from the paper’s full text.

TL;DR

iSEE is a random forest predictor of mutation-induced binding affinity changes that combines 31 interface structure, evolution, and energy features. It performs well on its training data and an older blind set but, like other current methods, generalizes poorly to a newer large SKEMPI 2.0 blind set.

Key findings

  • Trained on 1102 single point mutations in 57 protein–protein complexes from SKEMPI 1.1/DACUM, iSEE achieved cross-validated PCC 0.80 ± 0.06 and RMSE 1.41 ± 0.14 kcal/mol using only 31 features.
  • On the independent NM dataset of 19 mutations in one complex (1IAR), iSEE had RMSE 1.37 kcal/mol and PCC 0.73, ranking among the top four predictors with BindProfX (0.81), FoldX (0.72), and ZeMu (0.70).
  • On the blind S487 dataset from SKEMPI 2.0 (487 mutations in 56 complexes), no tested predictor performed well (PCC < 0.42); BindProfX had RMSE 1.20 kcal/mol and PCC 0.41, while iSEE had RMSE 1.32 kcal/mol and PCC 0.25. Averaging predictions improved performance to RMSE 1.18 kcal/mol and PCC 0.43.
  • Feature importance showed PSSM_wt and PSSM_diff were most important, followed by Eelec_diff and PSSM_IC; a full MDM2–p53 interface scan identified Y67 on MDM2 and F19 and W23 on p53 as sensitive sites, with F19 and W23 experimentally validated hot-spots.

Methods

  • Datasets: training used 1102 single point interface mutations in 57 dimeric complexes from DACUM/SKEMPI 1.1; independent tests used 19 mutations in 1IAR from the Benedix et al. NM dataset and 487 mutations in 56 complexes from SKEMPI 2.0; MDM2–p53 case study used 33 new experimental mutations, 7 at the detection limit. Only single point mutations in dimeric complexes were considered.
  • Features: 31 features from HADDOCK-refined wildtype and mutant structures—intermolecular van der Waals, electrostatic, desolvation, and buried surface area terms for wildtype and mutant-minus-wildtype differences—plus PSSM conservation features (PSSM_AA, PSSM_IC, PSSM_wt, PSSM_diff).
  • Analysis: random forest trained with R Caret, optimized by 10 times 10-fold cross-validation over ntree 10–100 and mtry 1–20; best model used ntree = 80 and mtry = 7. Performance was evaluated by RMSE and PCC, compared with FoldX, ZeMu, CC/PBSA, pred1, pred2, BeAtMuSiC, mCSM, and BindProfX; feature importance used mean squared error decrease over trees.

Limitations

  • The paper states that limited experimental data make it important to use few features to avoid overfitting; iSEE uses 31 features, whereas methods such as mCSM may use over 100. It also states that only single point mutations in dimeric complexes were considered for ease of computation, though the scheme can be extended to multimers.
  • The paper states that iSEE requires structures of both wildtype and mutant complexes, unlike mCSM, which needs only wildtype structures. It also notes that alignment depth did not appear to affect performance much but that a more systematic study is needed, and that using top-ranked versus top-four averaged HADDOCK models had no significant impact but needs more systematic analysis.
  • Evident from the text: generalization to new complexes is limited—on the large blind S487 dataset iSEE reached only PCC 0.25, and on the MDM2–p53 case all predictors had PCC below 0.40. The NM blind test is also very small, with only 19 mutations in one complex, and the MDM2–p53 experimental set includes 7 mutations at the detection limit.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/prot.25630

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 28 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    On the S487 dataset of 487 mutations in 56 protein complexes from SKEMPI 2.0, none of the current methods perform well, with PCC < 0.42.

    “Predictions for a new dataset of 487 mutations in 56 protein complexes from the recently published SKEMPI 2.0 database reveals that none of the current methods perform well (PCC < 0.42), although their combination does improve the predictions.”
    Quote in the paperSubstantial claim
  • Findingown data

    On the NM test set, the top four predictors by PCC were BindProfX (0.81), iSEE (0.73), FoldX (0.72), and ZeMu (0.70), all with PCCs over 0.70.

    “belonging to the top four predictors with PCCs over 0.70: BindProfX (0.81), iSEE (0.73), FoldX (0.72), and ZeMu (0.70).”
    Quote in the paperSubstantial claim
  • Findingown data

    Combining current methods improves predictions on the S487 dataset.

    “Predictions for a new dataset of 487 mutations in 56 protein complexes from the recently published SKEMPI 2.0 database reveals that none of the current methods perform well (PCC < 0.42), although their combination does improve the predictions.”
    Quote in the paperNot a substantial claim
  • Findingown data

    iSEE performs as well for mutations corresponding to different changes in side-chain sizes.

    “The predictor performs as well for ALA and non‐ALA mutations, mutations inside and outside loops, and mutations corresponding to different changes in side‐chain sizes (Figure 2B,D).”
    Quote in the paperSubstantial claim
  • Findingown data

    In iSEE's random forest model, PSSM_wt and PSSM_diff are the two most important features for prediction performance.

    “The results (Figure 5) reveal that the PSSM value of the wildtype amino acid (PSSM_wt) and the difference of PSSM values between mutant and wildtype residues (PSSM_diff) are the two most important features.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 14
  • Classified as probably not a substantial claim. 4
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingown data

    Averaging the predictions of the ΔΔG predictors on the S487 dataset produced a combined predictor that outperformed all individual predictors, with RMSE 1.18 kcal/mol and PCC 0.43.

    “The resulting combined predictor outperforms all the individual predictors with improved RMSE (1.18 kcal/mol) and PCC (0.43) (Figure 4).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    On the independent NM test set, iSEE achieved an RMSE of 1.37 kcal/mol and a PCC of 0.73.

    “iSEE compares favorably with the eight other predictors considered here over the independent NM test set with a RMSE of 1.37 kcal/mol and a PCC of 0.73 (Figure 3)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On the NM test set, the ΔΔG predictions of the top four predictors are statistically significant with p values lower than 0.001, while the other predictors have p values ranging from 0.006 to 0.523.

    “The ΔΔG predictions of the top four predictors are statistically significant with p values lower than 0.001, while other predictors have larger p values ranging from 0.006 to 0.523.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On the S487 dataset, BindProfX performed best with an RMSE of 1.20 kcal/mol and PCC of 0.41, while iSEE achieved an RMSE of 1.32 kcal/mol and PCC of 0.25.

    “BindProfX performs best with an RMSE of 1.20 kcal/mol and PCC of 0.41 while iSEE achieves an RMSE of 1.32 kcal/mol and PCC of 0.25.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The MDM2-p53 dataset contains ΔΔG measurements for 33 new mutations, 7 of which reached the experimental detection limit, and the average experimental error was estimated to be less than 0.5 kcal/mol.

    “It contains ΔΔG measurements for 33 new mutations, 7 of which have reached the experimental detection limit (Supporting Information Table S8). The average experimental error for this dataset was estimated to be less than 0.5 kcal/mol.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In full computational mutation scanning of the MDM2-p53 interface, none of the predicted ΔΔG values was negative, indicating that no interface mutations were predicted to strengthen the interaction between MDM2 and p53.

    “None of the predicted ΔΔG values was negative, which indicates that no interface mutations were predicted to strengthen the interaction between MDM2 and p53.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The prediction that no interface mutations strengthen MDM2-p53 is largely consistent with experimental data, in which only six mutations stabilized the complex and only by very small amounts (≥ −0.4 kcal/mol).

    “This is largely consistent with the experimental data: Experimentally only six mutations were found to stabilize the complex, but only by very small amounts (≥ −0.4 kcal/mol, Supporting Information Table S8).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    HADDOCK has been consistently ranking among the top predictors and scorers in CAPRI.

    “HADDOCK21 is our in‐house docking program, which has been consistently ranking among the top predictors and scorers in CAPRI, a community‐wide experiment for the prediction of biomolecular interactions.22”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] Janin J, Henrick K, Moult J, et al. CAPRI: a critical assessment of PRedicted interactions. Proteins Struct Funct Bioinform. 2003; 52(1): 2‐ 9. 12784359 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Empirical energy-based methods usually take linear forms, often with only energy-based terms, and fail to exploit evolution information, which can limit their ability to capture mutation effects on binding affinity.

    “They usually take a form of linear functions, often with only energy‐based terms, and fail to exploit evolution information, which can limit their ability to capture mutation effects on binding affinity.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [6] Guerois R, Nielsen JE, Serrano L. Predicting changes in the stability of proteins and protein complexes: a study of more than 1000 mutations. J Mol Biol. 2002; 320(2): 369‐ 387. 12079393 source

      Not checked in the free run.

    • cites [7] Dourado DFAR, Flores SC. A multiscale approach to predicting affinity changes in protein‐protein interfaces. Proteins Struct Funct Bioinform. 2014; 82(10): 2681‐ 2690. 24975440 source

      Not checked in the free run.

    • cites [8] Benedix A, Becker CM, de Groot BL, Caflisch A, Böckmann RA. Predicting free energy changes using structural ensembles. Nat Methods. 2009; 6(1): 3‐ 4. 19116609 source

      Not checked in the free run.

    • cites [9] Li M, Petukh M, Alexov E, Panchenko AR. Predicting the impact of missense mutations on protein–protein binding affinity. J Chem Theory Comput. 2014; 10(4): 1770‐ 1780. 24803870 PMC3985714 source

      Not checked in the free run.

    • cites [10] Dehouck Y, Kwasigroch JM, Rooman M, Gilis D. BeAtMuSiC: prediction of changes in protein–protein binding affinity on mutations. Nucleic Acids Res. 2013; 41(W1): W333‐ W339. 23723246 PMC3692068 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    PSSM has been proven to provide crucial information in related topics such as binding site predictions and hot-spot predictions.

    “PSSM has been proven to provide crucial information in various related topics, such as binding site predictions37 and hot‐spot predictions.38”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [37] Walia RR, Xue LC, Wilkins K, El‐Manzalawy Y, Dobbs D, Honavar V. RNABindRPlus: a predictor that combines machine learning and sequence homology‐based methods to improve the reliability of predicted RNA‐binding residues in proteins Kurgan L, editor. PLoS One. 2014; 9(5): e97725. 24846307 PMC4028231 source

      Not checked in the free run.

    • cites [38] Moreira IS, Koukos PI, Melo R, et al. SpotOn: high accuracy identification of protein‐protein Interface hot‐spots. Sci Rep. 2017; 7(1): 8007. 28808256 PMC5556074 source

      Not checked in the free run.

  • The quote only partly states this claim.

    Study design and conductreasoning

    iSEE was designed to use only 31 interface Structure, Evolution, and Energy-based features to lower overfitting risk.

    “iSEE, uses a limited number of interface Structure, Evolution, and Energy‐based features for the prediction.”
    • Also stated in: § INTRODUCTION · plain
      “It uses a small number of features to lower the overfitting risk”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Using a small number of sensitive predictive features is vital to the generalization and robustness of machine learning methods due to limited experimental data.
    • Rests on an assumption: The SKEMPI 1.1 dataset is limited in size, so using too many features to train a model risks overfitting.

  • May combine two claims.

    Interpretationreasoning

    PSSM_wt captures the evolutionary conservation of a specific amino acid at the mutation position, and PSSM_diff captures its change after mutation.

    “They capture the evolutionary conservation of a specific amino acid at the mutation position and its change after mutation, respectively.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Missense mutations are often linked to various human diseases, such as cancer.

    “Missense mutations are often linked to various human diseases,1 such as cancer.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] Wang X, Wei X, Thijssen B, Das J, Lipkin SM, Yu H. Three‐dimensional reconstruction of protein networks provides insight into human genetic disease. Nat Biotechnol. 2012; 30(2): 159‐ 164. 22252508 PMC3708476 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The conservation measure used in those ΔΔG predictors is structure-based and relies on the availability of structural homologs.

    “However, since the conservation measure they used is structure‐based, relying on the availability of structural homologs,13, 19”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] Brender JR, Zhang Y. Predicting the effect of mutations on protein‐protein binding interactions through structure‐based Interface profiles Jernigan RL, editor. PLoS Comput Biol. 2015; 11(10): e1004494. 26506533 PMC4624718 source

      Not checked in the free run.

    • cites [19] Xiong P, Zhang C, Zheng W, Zhang Y. BindProfX: assessing mutation‐induced binding affinity change by protein Interface profiles with pseudo‐counts. J Mol Biol. 2017; 429(3): 426‐ 434. 27899282 PMC5963940 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    There is still plenty of room for further improvements in ΔΔG predictors.

    “This indicates there is still a plenty of room for further improvements.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The S487 dataset is representative of protein–protein interface mutations generally, so poor performance on it indicates room for improvement for ΔΔG predictors broadly.

  • May combine two claims.

    Interpretationown data

    Averaging predictions from different ΔΔG predictors improved prediction performance, indicating that the predictors might use complementary features.

    “Interestingly, averaging the predictions from the different ΔΔG predictors generated an improved prediction performance, indicating the various ΔΔG predictors might use complementary features.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    F19 and W23 on p53 are experimental hot-spots.

    “The latter two are experimental hot‐spots41 (Supporting Information Tables S2 and S8) and the third one is a candidate for experimental verification.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [41] Bogan AA, Thorn KS. Anatomy of hot spots in protein interfaces. J Mol Biol. 1998; 280(1): 1‐ 9. 9653027 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    iSEE uses 31 features in total, a rather small number compared with other machine learning methods such as mCSM, which minimizes the risk of overfitting.

    “Compared with other machine learning methods, our predictor uses a rather small number of features, 31 in total which minimize the risk of overfitting (mCSM, eg, could use over 100 features).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    By combining structural, evolutionary, and energetic features and training on a large and diverse dataset, iSEE demonstrated consistent and high performance on various types of mutations during training and competed with state-of-the-art methods on independent blind test datasets.

    “By combining structural, evolutionary, and energetic features and training on a large and diverse dataset, our iSEE predictor not only demonstrated a consistent and high performance on various types of mutations during training, but also competed with state‐of‐the‐art methods (based on empirical potential or machine learning models) on independent blind test datasets.”
    Quote in the paperSubstantial claim
19 of 19

  • The paper relies on this without support.

    Using a small number of sensitive predictive features is vital to the generalization and robustness of machine learning methods due to limited experimental data.

    “Due to the limited number of experimental data, using a small number of sensitive predictive features is vital to the generalization and robustness of such machine learning methods.”

  • The paper relies on this without support.

    Experimental methods for measuring binding affinities are labor-intensive and time-consuming and lag behind sequencing technologies.

    “Although they provide valuable information, experimental methods can be labor‐intensive and time‐consuming, and, as a consequence, lag behind the rapid advances of sequencing technologies, which are generating a huge amount of data on disease‐causing mutations.”

  • The paper relies on this without support.

    The SKEMPI 1.1 dataset is limited in size, so using too many features to train a model risks overfitting.

    “However, the SKEMPI 1.1 dataset is still rather limited in size and one has to be careful not to use too many features to train a model to avoid overfitting problems.”

  • The paper relies on this without support.

    Insufficient conformational sampling, especially for mutations in flexible regions, can limit the accuracy of energy-based methods.

    “Insufficient conformational sampling, especially for mutations in flexible regions, can limit the accuracy of energy‐based methods.”

  • The paper relies on this without support.

    The S487 dataset is representative of protein–protein interface mutations generally, so poor performance on it indicates room for improvement for ΔΔG predictors broadly.

    “This indicates there is still a plenty of room for further improvements.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The final dataset for building mCSM-PPI2 includes 8338 single-point mutations, an increase of up to three-fold compared to previous methods that used data from the first version of SKEMPI.

    “Therefore, the final dataset for building mCSM-PPI2 predictive model includes 8338 single-point mutations (S8338), which represents an increase of up to three-fold in datapoints in comparison with previous methods that used data from the first version of SKEMPI with 2007 (S2007) (20,23), 1964 (S1964) (25), 1102 (S1102) (24) and 1327 (S1327) mutations (41).”

    From mCSM-PPI2: predicting the effects of mutations on protein–protein interactions (2019) · cites this paper as [24]

  • Significant efforts have been devoted to computationally study the effects of mutations on protein complexes, but their poor predictive performance on new variants, particularly mutations that lead to increased binding affinity, has limited their use.

    “Since then, significant efforts have been devoted to computationally study the effects of mutations on protein complexes (24,25) but their poor predictive performance on new variants, particularly mutations that lead to increased binding affinity of the complex, has limited their use.”

    From mCSM-PPI2: predicting the effects of mutations on protein–protein interactions (2019) · cites this paper as [24]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Geng, C., Vangone, A., Folkers, G. E., Xue, L. C., & Bonvin, A. M. J. J.
Affiliations
Utrecht University (NL)
Corresponding author
  • Xue, Li C.
  • Bonvin, Alexandre M. J. J.
ORCID
Venue
Proteins: Structure, Function, and Bioinformatics
Publisher
Wiley
Published
2018-12-03
Volume
87
Issue
2
Pages
110-119
ISSN
  • 0887-3585 (ISSN-L)
  • 1097-0134
DOI
10.1002/prot.25630
PMC
PMC6587874
OpenAlex
W2949787032
Cited by
82
References
43
Citations by year
2026: 3 · 2025: 9 · 2024: 13 · 2023: 15 · 2022: 12 · 2021: 9 · 2020: 15 · 2019: 6
FWCI
2.67
Type
journal article
Language
en
Open access
Licence
Locations
  • Proteins Structure Function and Bioinformatics (publishedVersion, cc-by-nc)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
Funders
China Scholarship Council (201406220132)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
Keywords
point mutations, protein-protein interfaces, ISEE, random forest, energy-based features, evolutionary conservation, protein-protein complex, mutation scanning, protein engineering, drug design
MeSH terms
Machine Learning, Protein Domains, Binding, Competitive, Models, Molecular, Mutation, Protein Binding, Proteins, Thermodynamics, Tumor Suppressor Protein p53, Evolution, Molecular, Computational Biology, Proto-Oncogene Proteins c-mdm2

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review