Evaluation of template‐based modeling in CASP13
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Performance in the template‐based modeling (TBM) category of CASP13 is assessed here, using a variety of metrics. Performance of the predictor groups that participated is ranked using the primary ranking score that was developed by the assessors for CASP12. This reveals that the best results are obtained by groups that include contact predictions or inter‐residue distance predictions derived from deep multiple sequence alignments. In cases where there is a good homolog in the wwPDB (TBM‐easy category), the best results are obtained by modifying a template. However, for cases with poorer homologs (TBM‐hard), very good results can be obtained without using an explicit template, by deep learning algorithms trained on the wwPDB. Alternative metrics are introduced, to allow testing of aspects of structural models that are not addressed by traditional CASP metrics. These include comparisons to the main‐chain and side‐chain torsion angles of the target, and the utility of models for solving crystal structures by the molecular replacement method. The alternative metrics are poorly correlated with the traditional metrics, and it is proposed that modeling has reached a sufficient level of maturity that the best models should be expected to satisfy this wider range of criteria.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
UCSF ChimeraX: Meeting modern challenges in visualization and analysis
Goddard, T. D., Huang, C. C., Meng, E. C., Pettersen, E. F., Couch, G. S., Morris, J. H., & Ferrin, T. E. (2018) · Protein Science · doi:10.1002/pro.3235 · Cited by 5,267
Big Data: Astronomical or Genomical?
Stephens, Z. D., Lee, S. Y., Faghri, F., Campbell, R. H., Zhai, C., Efron, M. J., Iyer, R., Schatz, M. C., Sinha, S., & Robinson, G. E. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002195 · Cited by 1,340
LGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069 · cited 1×
“GDT_HA is the high-accuracy version of the Global Distance Test, which assesses the overall fold in a way that gives greater reward for parts of the target reproduced with high precision.(23) The local difference distance test, lDDT, evaluates how well models reproduce an all-atom distance map.”
MethodA new clustering of antibody CDR loop conformations
North, B., Lehmann, A., & Dunbrack, R. L. (2011) · Journal of Molecular Biology · doi:10.1016/j.jmb.2010.10.030 · Cited by 380 · cited 2×
“The score for a given dihedral was defined based on the metric previously used for protein dihedral analysis by North et al 26 (the squared length of the chord on a unit circle resulting from the angular deviation from the target), normalized to the range (0 … 1): For each residue present in both…”
MethodHaas, J., Barbato, A., Behringer, D., Studer, G., Roth, S., Bertoni, M., Mostaguir, K., Gumienny, R., & Schwede, T. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25431 · Cited by 168 · cited 1×
“31 In these scripts, a full MR search is not carried out; rather, the model is superimposed on each copy Since CASP7, it has become much more common for predictors to submit local error estimates in the B-factor fields of submitted models.”
Background
Cited by
Improved protein structure prediction using potentials from deep learning
Senior, A. W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., Žídek, A., Nelson, A. W. R., Bridgland, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1923-7 · Cited by 3,378
Zhou, X., Zheng, W., Li, Y., Pearce, R., Zhang, C., Bell, E. W., Zhang, G., & Zhang, Y. (2022) · Nature Protocols · doi:10.1038/s41596-022-00728-0 · Cited by 519
Critical assessment of methods of protein structure prediction (CASP)—Round XII
Moult, J., Fidelis, K., Kryshtafovych, A., Schwede, T., & Tramontano, A. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25415 · Cited by 326
Self-citationHigh‐accuracy protein structure prediction in CASP14
Pereira, J., Simpkin, A. J., Hartmann, M. D., Rigden, D. J., Keegan, R. M., & Lupas, A. N. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26171 · Cited by 314
“…other terms to evaluate the geometric fitting of models: the (1) backbone and (2) side-chain deviation scores, which measure the local difference of torsion angles in the model relative to the target 15 ; and (3) the MolProbity clashscore, which assesses the number of serious clashes in the model.”
BackgroundMethodInfluentialProMod3—A versatile homology modelling toolbox
Studer, G., Tauriello, G., Bienert, S., Biasini, M., Johner, N., & Schwede, T. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1008667 · Cited by 294
The chemistry of snake venom and its medicinal potential
Oliveira, A. L., Viegas, M. F., da Silva, S. L., Soares, A. M., Ramos, M. J., & Fernandes, P. A. (2022) · Nature Reviews Chemistry · doi:10.1038/s41570-022-00393-7 · Cited by 289
Haas, J., Barbato, A., Behringer, D., Studer, G., Roth, S., Bertoni, M., Mostaguir, K., Gumienny, R., & Schwede, T. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25431 · Cited by 168
Implications of AlphaFold 2 for crystallographic phasing by molecular replacement
McCoy, A. J., Sammito, M. D., & Read, R. J. (2022) · Acta Crystallographica Section D Structural Biology · doi:10.1107/s2059798321012122 · Cited by 104
Self-citationPutting AlphaFold models to work with phenix.process_predicted_model and ISOLDE
Oeffner, R. D., Croll, T. I., Millán, C., Poon, B. K., Schlicksup, C. J., Read, R. J., & Terwilliger, T. C. (2022) · Acta Crystallographica Section D Structural Biology · doi:10.1107/s2059798322010026 · Cited by 63
Self-citationAssessing the utility of CASP14 models for molecular replacement
Millán, C., Keegan, R. M., Pereira, J., Sammito, M. D., Simpkin, A. J., McCoy, A. J., Lupas, A. N., Hartmann, M. D., Rigden, D. J., & Read, R. J. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26214 · Cited by 62
Self-citationSimpkin, A. J., Thomas, J. M. H., Keegan, R. M., & Rigden, D. J. (2022) · Acta Crystallographica Section D Structural Biology · doi:10.1107/s2059798322003576 · Cited by 24
Oda, T. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26551 · Cited by 17
(2022) · doi:10.1099/jgv.0.001716 · Cited by 8
Metadata
- Authors
- Croll, T. I., Sammito, M. D., Kryshtafovych, A., & Read, R. J.
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2019-08-20
- Volume
- 87
- Issue
- 12
- Pages
- 1113-1127
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- OpenAlex
- W2967999860
- Cited by
- 66
- References
- 5
- FWCI
- 3.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.