Assessment of protein model structure accuracy estimation in CASP14 : Old and new challenges
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
In CASP, blind testing of model accuracy estimation methods has been conducted on models submitted by tertiary structure prediction servers. In CASP14, model accuracy estimation results were evaluated in terms of both global and local structure accuracy, as in the previous CASPs. Unlike the previous CASPs that did not show pronounced improvements in performance, the best single‐model method (from the Baker group) showed an improved performance in CASP14, particularly in evaluating global structure accuracy when compared to both the best single‐model methods in previous CASPs and the best multi‐model methods in the current CASP. Although the CASP14 experiment on model accuracy estimation did not deal with the structures generated by AlphaFold2, new challenges that have arisen due to the success of AlphaFold2 are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Mariani, V., Biasini, M., Barbato, A., & Schwede, T. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt473 · Cited by 1,138
LGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069 · cited 1×
“When models are accurate, the choice of evaluation measure, such as GDT-TS 12 or LDDT, 13 is less important.”
MethodHigh‐accuracy protein structure prediction in CASP14
Pereira, J., Simpkin, A. J., Hartmann, M. D., Rigden, D. J., Keegan, R. M., & Lupas, A. N. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26171 · Cited by 314
Improved protein structure refinement guided by deep learning based accuracy estimation
Hiranuma, N., Park, H., Baek, M., Anishchenko, I., Dauparas, J., & Baker, D. (2021) · Nature Communications · doi:10.1038/s41467-021-21511-x · Cited by 243 · cited 1×
“The best EMA method in CASP14 was from the Baker group, 11 and it also performed better than other methods in CASP-COVID 19 .”
Method
Research works printed but not linked: 15
- 10.1002/prot.21669 · DOI · Paper page
- 10.1002/prot.22534 · DOI · Paper page
- 10.1002/prot.23180 · DOI · Paper page
- 10.1002/prot.24347 · DOI · Paper page
- 10.1002/prot.24919 · DOI · Paper page
- 10.1002/prot.25371 · DOI · Paper page
- 10.1002/prot.25804 · DOI · Paper page
- Kinch LN, 2021, Topology evaluation of models for difficult targets in the 14th round of the critical assessment of protein structure prediction (CASP14), Proteins Struct Funct Bioinformatics
- 10.1002/prot.24172 · DOI · Paper page
- 10.1093/nar/gku294 · DOI
- 10.1002/prot.24399 · DOI · Paper page
- 10.1016/j.bpj.2011.09.012 · DOI · Paper page
- 10.1002/prot.25775 · DOI · Paper page
- 10.1002/prot.26231 · DOI
- 10.1002/prot.26222 · DOI · Paper page
Cited by
Critical assessment of methods of protein structure prediction (CASP)—Round XIV
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26237 · Cited by 503
Self-citationState-of-the-Art Estimation of Protein Model Accuracy Using AlphaFold
Roney, J. P., & Ovchinnikov, S. (2022) · Physical Review Letters · doi:10.1103/physrevlett.129.238101 · Cited by 281
Assessment of three‐dimensional RNA structure prediction in CASP15
Das, R., Kretsch, R. C., Simpkin, A. J., Mulvaney, T., Pham, P., Rangan, R., Bu, F., Keegan, R. M., Topf, M., Rigden, D. J., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26602 · Cited by 139
Critical assessment of methods of protein structure prediction ( CASP )—Round XV
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26617 · Cited by 148
Self-citationNew prediction categories in CASP15
Kryshtafovych, A., Antczak, M., Szachniuk, M., Zok, T., Kretsch, R. C., Rangan, R., Pham, P., Das, R., Robin, X., Studer, G., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26515 · Cited by 54
“48 It has attracted the attention of many developers, with over 70 methods tested in the previous CASP experiment.”
BackgroundMethodSelf-citation
Metadata
- Authors
- Kwon, S., Won, J., Kryshtafovych, A., & Seok, C.
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2021-08-05
- Volume
- 89
- Issue
- 12
- Pages
- 1940-1948
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- OpenAlex
- W3186243460
- Cited by
- 57
- References
- 5
- FWCI
- 3.15
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- National Institute of General Medical Sciences (R01GM100482)
- National Research Foundation of Korea (2019M3E5D4066898, 2020M3A9G7103933)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.