PON-tstab: Protein Variant Stability Predictor. Importance of Training Data Quality
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Several methods have been developed to predict effects of amino acid substitutions on protein stability. Benchmark datasets are essential for method training and testing and have numerous requirements including that the data is representative for the investigated phenomenon. Available machine learning algorithms for variant stability have all been trained with ProTherm data. We noticed a number of issues with the contents, quality and relevance of the database. There were errors, but also features that had not been clearly communicated. Consequently, all machine learning variant stability predictors have been trained on biased and incorrect data. We obtained a corrected dataset and trained a random forests-based tool, PON-tstab, applicable to variants in any organism. Our results highlight the importance of the benchmark quality, suitability and appropriateness. Predictions are provided for three categories: stability decreasing, increasing and those not affecting stability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
UniProt: the universal protein knowledgebase
The UniProt Consortium (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw1099 · Cited by 4,436
Assessing the accuracy of prediction algorithms for classification: an overview
Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., & Nielsen, H. (2000) · Bioinformatics · doi:10.1093/bioinformatics/16.5.412 · Cited by 2,111
Pires, D. E. V., Ascher, D. B., & Blundell, T. L. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku411 · Cited by 932
Potapov, V., Cohen, M., & Schreiber, G. (2009) · Protein Engineering Design and Selection · doi:10.1093/protein/gzp030 · Cited by 349
Cited by
Machine-learning-guided directed evolution for protein engineering
Yang, K. K., Wu, Z., & Arnold, F. H. (2019) · Nature Methods · doi:10.1038/s41592-019-0496-6 · Cited by 1,130
Machine Learning in Enzyme Engineering
Mazurenko, S., Prokop, Z., & Damborsky, J. (2020) · ACS Catalysis · doi:10.1021/acscatal.9b04321 · Cited by 410
PremPS: Predicting the impact of missense mutations on protein stability
Chen, Y., Lu, H., Zhang, N., Zhu, Z., Wang, S., & Li, M. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1008543 · Cited by 260
Rapid protein stability prediction using deep learning representations
Blaabjerg, L. M., Kassem, M. M., Good, L. L., Jonsson, N., Cagiada, M., Johansson, K. E., Boomsma, W., Stein, A., & Lindorff-Larsen, K. (2023) · eLife · doi:10.7554/elife.82593 · Cited by 177
Montanucci, L., Capriotti, E., Frank, Y., Ben-Tal, N., & Fariselli, P. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2923-1 · Cited by 167
ThermoMutDB: a thermodynamic database for missense mutations
Xavier, J. S., Nguyen, T.-B., Karmarkar, M., Portelli, S., Rezende, P. M., Velloso, J. P. L., Ascher, D. B., & Pires, D. E. V. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa925 · Cited by 101
“However, this database has not been updated for 7 years and many errors have been identified previously (29,30), limiting both previous methods and future developments.”
MethodOn the biases in predictions of protein stability changes upon variations: the INPS test case
Montanucci, L., Savojardo, C., Martelli, P. L., Casadio, R., & Fariselli, P. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty979 · Cited by 33
“…it must be stressed that the correlations reported in Table 1 are only intended to assess the anti-symmetricity of the methods, and not as estimators of their overall performances, which is a very difficult task that also requires an evaluation of stability of the dataset (Yang et al. , 2018).”
MethodPennica, C., Hanna, G., Islam, S. A., Sternberg, M. J. E., & David, A. (2023) · Journal of Molecular Biology · doi:10.1016/j.jmb.2023.168060 · Cited by 25
Metadata
- Authors
- Yang, Y., Urolagin, S., Niroula, A., Ding, X., Shen, B., & Vihinen, M.
- Venue
- International Journal of Molecular Sciences
- Publisher
- MDPI AG
- Published
- 2018-03-28
- Volume
- 19
- Issue
- 4
- Pages
- 1009
- ISSN
- 1422-0067 (ISSN-L)
- OpenAlex
- W2794541436
- Cited by
- 74
- References
- 5
- FWCI
- 3.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.