Assessing computational methods for predicting protein stability upon mutation: good on average but not in the details
Abstract
Methods for protein modeling and design advanced rapidly in recent years. At the heart of these computational methods is an energy function that calculates the free energy of the system. Many of these functions were also developed to estimate the consequence of mutation on protein stability or binding affinity. In the current study, we chose six different methods that were previously reported as being able to predict the change in protein stability (DeltaDeltaG) upon mutation: CC/PBSA, EGAD, FoldX, I-Mutant2.0, Rosetta and Hunter. We evaluated their performance on a large set of 2156 single mutations, avoiding for each program the mutations used for training. The correlation coefficients between experimental and predicted DeltaDeltaG values were in the range of 0.59 for the best and 0.26 for the worst performing method. All the tested computational methods showed a correct trend in their predictions, but failed in providing the precise values. This is not due to lack in precision of the experimental data, which showed a correlation coefficient of 0.86 between different measurements. Combining the methods did not significantly improve prediction accuracy compared to a single method. These results suggest that there is still room for improvement, which is crucial if we want forcefields to perform better in their various tasks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
3D-structure-based ΔΔG predictors show modest performance even using 3D structures from PDB, with correlation of 0.59 or less in independent tests.
“the 3D-structure based ΔΔG predictors show modest performance even using 3D structures from PDB [26], with correlation of 0.59 or less in independent tests [27].”
From Using AlphaFold to predict the impact of single mutations on protein stability and function (2023) · cites this paper as [27]
Cites
Guerois, R., Nielsen, J. E., & Serrano, L. (2002) · Journal of Molecular Biology · doi:10.1016/s0022-2836(02)00442-4 · Cited by 1,728 · see citing works
Predicting free energy changes using structural ensembles
Benedix, A., Becker, C. M., de Groot, B. L., Caflisch, A., & Böckmann, R. A. (2009) · Nature Methods · doi:10.1038/nmeth0109-3 · Cited by 234 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep mutational scanning: a new style of protein science
Fowler, D. M., & Fields, S. (2014) · Nature Methods · doi:10.1038/nmeth.3027 · Cited by 1,287 · see citing works
DynaMut: predicting the impact of mutations on protein conformation, flexibility and stability
Rodrigues, C. H., Pires, D. E., & Ascher, D. B. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky300 · Cited by 1,086 · see citing works
Using AlphaFold to predict the impact of single mutations on protein stability and function
Pak, M. A., Markhieva, K. A., Novikova, M. S., Petrov, D. S., Vorobyev, I. S., Maksimova, E. S., Kondrashov, F. A., & Ivankov, D. N. (2023) · PLOS One · doi:10.1371/journal.pone.0282689 · Cited by 322 · see citing works
Araya, C. L., Fowler, D. M., Chen, W., Muniez, I., Kelly, J. W., & Fields, S. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1209751109 · Cited by 230 · see citing works
Computational prediction of protein–protein binding affinities
Siebenmorgen, T., & Zacharias, M. (2020) · WIREs Computational Molecular Science · doi:10.1002/wcms.1448 · Cited by 184 · see citing works
“101,102 For protein engineering, it is of significant interest to predict changes in binding affinity of protein – protein complexes due to mutations and to identify important residues for the interaction (hotspots).”
BackgroundQuantification of biases in predictions of protein stability changes upon mutations
Pucci, F., Bernaerts, K. V., Kwasigroch, J. M., & Rooman, M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty348 · Cited by 169 · see citing works
Towards precision medicine: advances in computational approaches for the analysis of human variants
Peterson, T. A., Doughty, E., & Kann, M. G. (2013) · Journal of Molecular Biology · doi:10.1016/j.jmb.2013.08.008 · Cited by 123 · see citing works
Assessing predictors of changes in protein stability upon mutation using self-consistency
Thiltgen, G., & Goldstein, R. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0046084 · Cited by 110 · see citing works
Usmanova, D. R., Bogatyreva, N. S., Ariño Bernad, J., Eremina, A. A., Gorshkova, A. A., Kanevskiy, G. M., Lonishin, L. R., Meister, A. V., Yakupova, A. G., Kondrashov, F. A., et al. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty340 · Cited by 93 · see citing works
Using AlphaFold to predict the impact of single mutations on protein stability and function
Pak, M. A., Markhieva, K. A., Novikova, M. S., Petrov, D. S., Vorobyev, I. S., Maksimova, E. S., Kondrashov, F. A., & Ivankov, D. N. (2021) · doi:10.1101/2021.09.19.460937 · Cited by 322 · see citing works
Free energy perturbation calculations of the thermodynamics of protein side-chain mutations
Steinbrecher, T., Abel, R., Clark, A., & Friesner, R. (2017) · Journal of Molecular Biology · doi:10.1016/j.jmb.2017.03.002 · Cited by 46 · see citing works
Problems in variation interpretation guidelines and in their implementation in computational tools
Vihinen, M. (2020) · Molecular Genetics & Genomic Medicine · doi:10.1002/mgg3.1206 · Cited by 22 · see citing works
- Authors
- Potapov, V., Cohen, M., & Schreiber, G.
- Affiliations
- Weizmann Institute of Science (IL)
- Corresponding author
- Schreiber, G.
- Venue
- Protein Engineering Design and Selection
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-06-26
- Volume
- 22
- Issue
- 9
- Pages
- 553-560
- ISSN
- 0269-2139 (ISSN-L)
- 1741-0126
- 1741-0134
- OpenAlex
- W2168749685
- Cited by
- 349
- References
- 2
- Citations by year
- 2026: 7 · 2025: 11 · 2024: 7 · 2023: 19 · 2022: 19 · 2021: 33 · 2020: 22 · 2019: 23 · 2018: 18 · 2017: 17 · 2016: 25 · 2015: 28 · 2014: 27 · 2013: 20 · 2012: 35
- FWCI
- 9.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Protein Engineering Design and Selection (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein stability prediction, point mutations, ΔΔG, energy function, binding affinity prediction, FoldX, Rosetta, hunters, method comparison, force fields, protein design
- MeSH terms
- Algorithms, Mutation, Proteins, Thermodynamics, Protein Engineering, Linear Models, Computational Biology, Databases, Protein, Protein Stability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
