Methods of model accuracy estimation can help selecting the best models from decoy sets: Assessment of model accuracy estimations in CASP 11
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
The article presents assessment of the model accuracy estimation methods participating in CASP11. The results of the assessment are expected to be useful to both—developers of the methods and users who way too often are presented with structural models without annotations of accuracy. The main emphasis is placed on the ability of techniques to identify the best models from among several available. Bivariate descriptive statistics and ROC analysis are used to additionally assess the overall correctness of the predicted model accuracy scores, the correlation between the predicted and observed accuracy of models, the effectiveness in distinguishing between good and bad models, the ability to discriminate between reliable and unreliable regions in models, and the accuracy of the coordinate error self‐estimates. A rigid‐body measure (GDT_TS) and three local‐structure‐based scores (LDDT, CADaa, and SphereGrinder) are used as reference measures for evaluating methods' performance. Consensus methods, taking advantage of the availability of several models for the same target protein, perform well on the majority of tasks. Methods that predict accuracy on the basis of a single model perform comparably to consensus methods in picking the best models and in the estimation of how accurate is the local structure. More groups than in previous experiments submitted reasonable error estimates of their own models, most likely in response to a recommendation from CASP and the increasing demand from users. Proteins 2016; 84(Suppl 1):349–369. © 2015 Wiley Periodicals, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mariani, V., Biasini, M., Barbato, A., & Schwede, T. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt473 · Cited by 1,138
Self-citationLGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069
Cited by
The SWISS-MODEL Repository—new features and functionality
Bienert, S., Waterhouse, A., de Beer, T. A. P., Tauriello, G., Studer, G., Bordoli, L., & Schwede, T. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkw1132 · Cited by 1,745
Self-citationQMEANDisCo—distance constraints applied on model quality estimation
Studer, G., Rempfer, C., Waterhouse, A. M., Gumienny, R., Haas, J., & Schwede, T. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz828 · Cited by 1,032
Self-citationCritical assessment of methods of protein structure prediction (CASP)—Round XII
Moult, J., Fidelis, K., Kryshtafovych, A., Schwede, T., & Tramontano, A. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25415 · Cited by 326
Self-citationNew prediction categories in CASP15
Kryshtafovych, A., Antczak, M., Szachniuk, M., Zok, T., Kretsch, R. C., Rangan, R., Pham, P., Das, R., Robin, X., Studer, G., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26515 · Cited by 54
Self-citationChallenges in bridging the gap between protein structure prediction and functional interpretation
Varadi, M., Tsenkov, M., & Velankar, S. (2025) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26614 · Cited by 36
Accurate protein stability predictions from homology models
Valanciute, A., Nygaard, L., Zschach, H., Maglegaard Jepsen, M., Lindorff-Larsen, K., & Stein, A. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.11.048 · Cited by 33
“A number of tools have been developed to assess the quality of an individual homology model based on similarity to experimentally resolved structures in terms of torsion angles, core packing and other high-resolution properties [18,19].”
BackgroundAssessing protein model quality based on deep graph coupled networks using protein language model
(2023) · doi:10.1093/bib/bbad420 · Cited by 18
Metadata
- Authors
- Kryshtafovych, A., Barbato, A., Monastyrskyy, B., Fidelis, K., Schwede, T., & Tramontano, A.
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2015-09-28
- Volume
- 84
- Issue
- S1
- Pages
- 349-369
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- OpenAlex
- W2168321064
- Cited by
- 68
- References
- 2
- FWCI
- 5.15
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.