Tertiary structure assessment at CASP15
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
The results of tertiary structure assessment at CASP15 are reported. For the first time, recognizing the outstanding performance of AlphaFold 2 (AF2) at CASP14, all single‐chain predictions were assessed together, irrespective of whether a template was available. At CASP15, there was no single stand‐out group, with most of the best‐scoring groups—led by PEZYFoldings, UM‐TBM, and Yang Server—employing AF2 in one way or another. Many top groups paid special attention to generating deep Multiple Sequence Alignments (MSAs) and testing variant MSAs, thereby allowing them to successfully address some of the hardest targets. Such difficult targets, as well as lacking templates, were typically proteins with few homologues. Local divergence between prediction and target correlated with localization at crystal lattice or chain interfaces, and with regions exhibiting high B‐factor factors in crystal structure targets, and should not necessarily be considered as representing error in the prediction. However, analysis of exposed and buried side chain accuracy showed room for improvement even in the latter. Nevertheless, a majority of groups produced high‐quality predictions for most targets, which are valuable for experimental structure determination, functional analysis, and many other tasks across biology. These include those applying methods similar to those used to generate major resources such as the AlphaFold Protein Structure Database and the ESM Metagenomic atlas: the confidence estimates of the former were also notably accurate.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkab1061 · Cited by 7,850
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
Burley, S. K., Bhikadiya, C., Bi, C., Bittrich, S., Chen, L., Crichlow, G. V., Christie, C. H., Dalenberg, K., Di Costanzo, L., Duarte, J. M., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkaa1038 · Cited by 1,581 · cited 1×
“Even in the earliest CASP exercises, simpler targets — those for which a homologous structure could be identified in the Protein Data Bank 3 — were modeled quite well.”
BackgroundMariani, V., Biasini, M., Barbato, A., & Schwede, T. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt473 · Cited by 1,138
LGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069 · cited 1×
“17 In the third group are GDT_HA , the high-accuracy variant of the Global Distance Test — Total Score (GDT_TS), which measures global fold accuracy 18 and ASE , the Accuracy Self Estimate, measuring the correlation between error estimates and actual model errors.”
A large‐scale experiment to assess protein structure prediction methods
Moult, J., Pedersen, J. T., Judson, R., & Fidelis, K. (1995) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.340230303 · Cited by 555
Critical assessment of methods of protein structure prediction (CASP)—Round XIV
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26237 · Cited by 503
Self-citation
Research works printed but not linked: 42
- 10.1002/prot.340230317 · DOI
- 10.1038/s41592-020-0848-2 · DOI · Paper page
- 10.1016/j.cbpa.2021.04.005 · DOI · Paper page
- 10.1002/prot.26172 · DOI · Paper page
- 10.1038/s43588-022-00373-3 · DOI · Paper page
- KryshtafovychA RigdenD.To split or not to split: CASP15 targets and their processing into tertiary structure evaluation units. Published online March 13 2023. doi:10.22541/au.167872023.39044035/v1 · DOI
- 10.1002/prot.24172 · DOI · Paper page
- 10.1002/prot.24399 · DOI · Paper page
- 10.1002/prot.25800 · DOI · Paper page
- 10.1107/S0907444909042073 · DOI · Paper page
- 10.1107/S2052252517008466 · DOI
- 10.1002/prot.25425 · DOI · Paper page
- 10.1002/prot.22488 · DOI · Paper page
- Zhou Y, 2016, Prediction of Protein Secondary Structure
- 10.1002/bip.360221211 · DOI · Paper page
- 10.1016/0022-2836(73)90011-9 · DOI · Paper page
- SimpkinAJ ElliottLG StevensonK KrissinelE RigdenD KeeganRM.Slice'N'Dice: maximising the value of predicted models for structural biologists.bioRxiv. Published online July 2 2022. doi:10.1101/2022.06.30.497974 · DOI
- Krissinel E, 2012, Enhanced fold recognition using efficient short fragment clustering, J Mol Biochem, 1, 76
- 10.1107/S0907444901012471 · DOI · Paper page
- 10.1073/pnas.1701640114 · DOI · Paper page
- Pedregosa F, 2011, Scikit‐learn: machine learning in python, J Mach Learn Res, 12, 2825
- 10.1107/S0907444996012255 · DOI · Paper page
- 10.1107/S2059798322007732 · DOI
- 10.1016/j.jmb.2006.02.053 · DOI · Paper page
- 10.1093/nar/gkw454 · DOI
- 10.1038/s41592-022-01488-1 · doi:10.1038/s41592-022-01488-1 · Paper page
- 10.1126/science.abj8754 · doi:10.1126/science.abj8754 · Paper page
- ZhengW WuyunQ FreddolinoPL ZhangY.Integrating deep learning with multi‐MSA and threading alignments for high quality protein monomer and complex structure prediction in CASP15.Proteins. 2023;91(12):1684‐1703. doi:10.1002/prot.26585 · DOI · Paper page
- 10.1038/s41596-022-00728-0 · DOI · Paper page
- PengZ WangW WeiH LiX YangJ.Improved protein structure prediction with trRosettaX2 AlphaFold2 and optimized MSAs in CASP15.Proteins. 2023;91(12):1704‐1711. doi:10.1002/prot.26570 · DOI
- 10.1126/science.ade2574 · doi:10.1126/science.ade2574 · Paper page
- OdaT.Improving protein structure prediction with extended sequence similarity searches and deep‐learning‐based refinement in CASP15. Published online April 17 2023. doi:10.22541/au.168170992.27078535/v1 · DOI · Paper page
- 10.1111/j.2397-2335.1922.tb00768.x · doi:10.1111/j.2397-2335.1922.tb00768.x · Paper page
- 10.1093/bioinformatics/btz679 · DOI · Paper page
- 10.1371/journal.pcbi.1003889 · DOI · Paper page
- 10.1016/j.str.2022.05.001 · doi:10.1016/j.str.2022.05.001
- 10.1093/gbe/evac069 · DOI
- StuderG TaurielloG SchwedeT.Assessment of the assessment—All about complexes.Proteins. 2023;91(12):1850‐1860. doi:10.1002/prot.26612 · DOI · Paper page
- 10.1007/978-94-024-1069-3 · DOI
- 10.1093/nar/gkh468 · DOI · Paper page
- Mulvaney T Kretsch RC Elliott L et al. CASP15 cryo‐EM protein and RNA targets: Refinement and analysis using experimental maps.Proteins.2023;91(12):1935‐1951. doi:10.1002/prot.26644 · DOI · Paper page
- WuT GuoZ ChengJ.Atomic protein structure refinement using all‐atom graph representations and SE(3)‐equivariant graph transformer.Bioinformatics. Published online May 5 2023. doi:10.1093/bioinformatics/btad298 · DOI
Cited by
Critical assessment of methods of protein structure prediction ( CASP )—Round XV
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26617 · Cited by 148
Self-citationOzden, B., Kryshtafovych, A., & Karaca, E. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26598 · Cited by 51
Self-citationBreaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15
Kryshtafovych, A., Montelione, G., Rigden, D., Mesdaghi, S., Karaca, E., & Moult, J. (2023) · doi:10.22541/au.169199504.48580490/v1 · Cited by 31
Self-citationBreaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15
Kryshtafovych, A., Montelione, G. T., Rigden, D. J., Mesdaghi, S., Karaca, E., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26584 · Cited by 31
Self-citationAssessment of Nucleic Acid Structure Prediction in CASP16
Kretsch, R. C., Hummer, A. M., He, S., Yuan, R., Zhang, J., Karagianes, T., Cong, Q., Kryshtafovych, A., & Das, R. (2026) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.70072 · Cited by 51
“Additionally, using a 62% sequence identity threshold, as used in the proteins assessment [32, 33] reduced all Neff values, but did not affect the reported trends.”
Self-citation
Metadata
- Authors
- Simpkin, A. J., Mesdaghi, S., Sánchez Rodríguez, F., Elliott, L., Murphy, D. L., Kryshtafovych, A., Keegan, R. M., & Rigden, D. J.
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2023-09-25
- Volume
- 91
- Issue
- 12
- Pages
- 1616-1635
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- OpenAlex
- W4387002923
- Cited by
- 47
- References
- 19
- FWCI
- 3.51
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Funders
- Biotechnology and Biological Sciences Research Council
- National Institute of General Medical Sciences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Tertiary structure assessment at CASP15. https://app.cochat.ai/papers/10.1002/prot.26593
- The paper
- Simpkin, A. J., Mesdaghi, S., Sánchez Rodríguez, F., Elliott, L., Murphy, D. L., Kryshtafovych, A., Keegan, R. M., & Rigden, D. J. (2023). Tertiary structure assessment at CASP15. Proteins: Structure, Function, and Bioinformatics. https://doi.org/10.1002/prot.26593
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.