New prediction categories in CASP15
Open accessPublished version of a preprint
Abstract
Prediction categories in the Critical Assessment of Structure Prediction (CASP) experiments change with the need to address specific problems in structure modeling. In CASP15, four new prediction categories were introduced: RNA structure, ligand‐protein complexes, accuracy of oligomeric structures and their interfaces, and ensembles of alternative conformations. This paper lists technical specifications for these categories and describes their integration in the CASP data management system.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Liebschner, D. C., Afonine, P. V., Baker, M. L., Bunkòczi, G., Chen, V. B., Croll, T. I., Hintze, B. J., Hung, L., Jain, S., McCoy, A. J., et al. (2019) · Acta Crystallographica Section D Structural Biology · doi:10.1107/s2059798319011471 · Cited by 6,709 · cited 1× · see citing works
“25 Given two sets of interactions, one for the model and 27,28 It is worth noting that model-independent measures are known to overinflate scores for compressed models with unrealistic geometries.”
BackgroundComparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · see citing works
A solution for the best rotation to relate two sets of vectors
Kabsch, W. (1976) · Acta Crystallographica Section A · doi:10.1107/s0567739476001873 · Cited by 3,462 · see citing works
Mariani, V., Biasini, M., Barbato, A., & Schwede, T. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt473 · Cited by 1,138 · cited 1× · see citing works
“Local predictions were compared against the per-residue lDDT 17 and CAD (AA-variant) 52 scores, which assess the accuracy of relative atom positions in the neighborhood, including neighboring chains.”
MethodSelf-citationGeometric nomenclature and classification of RNA base pairs
Leontis, N. B., & Westhof, É. (2001) · RNA · doi:10.1017/s1355838201002515 · Cited by 1,074 · cited 1× · see citing works
“25 Given two sets of interactions, one for the model and 27,28 It is worth noting that model-independent measures are known to overinflate scores for compressed models with unrealistic geometries.”
BackgroundLGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069 · cited 1× · see citing works
“19 These are commonly used measures in protein-CASP evaluation that are also adopted here for RNA evaluation.”
MethodBertoni, M., Kiefer, F., Biasini, M., Bordoli, L., & Schwede, T. (2017) · Scientific Reports · doi:10.1038/s41598-017-09654-8 · Cited by 791 · cited 2× · see citing works
“This is the oligomeric Template Modeling score (TM-score) 49 for overall topology (SCORE) and the contact based QS-score 50 which is interface centric (QSCORE).”
BackgroundSelf-citationBetter Informed Distance Geometry: Using What We Know To Improve Conformation Generation
Riniker, S., & Landrum, G. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00654 · Cited by 765 · cited 1× · see citing works
“38 Subsequently, the Mol objects are written to the MDL-formatted file, 33 which can be used as a ligand submission template.”
MethodDalby, A., Nourse, J. G., Hounshell, W. D., Gushurst, A. K. I., Grier, D., Leland, B., & Laufer, J. (1992) · Journal of chemical information and computer sciences · doi:10.1021/ci00007a012 · Cited by 600 · cited 1× · see citing works
“38 Subsequently, the Mol objects are written to the MDL-formatted file, 33 which can be used as a ligand submission template.”
MethodA large‐scale experiment to assess protein structure prediction methods
Moult, J., Pedersen, J. T., Judson, R., & Fidelis, K. (1995) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.340230303 · Cited by 555 · see citing works
Self-citationUS-align: universal structure alignments of proteins, nucleic acids, and macromolecular complexes
Zhang, C., Shine, M., Pyle, A. M., & Zhang, Y. (2022) · Nature Methods · doi:10.1038/s41592-022-01585-1 · Cited by 511 · cited 1× · see citing works
“This is the oligomeric Template Modeling score (TM-score) 49 for overall topology (SCORE) and the contact based QS-score 50 which is interface centric (QSCORE).”
BackgroundCritical assessment of methods of protein structure prediction (CASP)—Round XIII
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2019) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25823 · Cited by 506 · see citing works
Self-citationCritical assessment of methods of protein structure prediction (CASP)—Round <scp>XIV</scp>
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26237 · Cited by 503 · see citing works
Self-citationDockQ: A Quality Measure for Protein-Protein Docking Models
Basu, S., & Wallner, B. (2016) · PLOS ONE · doi:10.1371/journal.pone.0161879 · Cited by 460 · cited 1× · see citing works
“51 Notably, DockQ evaluates pairwise interfaces, necessitating the introduction of a weighted average metric — termed DockQ-wave — to effectively score higher-order complexes.”
BackgroundApplying and improving AlphaFold at CASP14
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26257 · Cited by 370 · cited 3× · see citing works
“…ideas and techniques with the protein prediction community, where deep learning methods recently made a significant impact on the modeling accuracy, 11,12 increase visibility of the field, and use CASP's standardized platform for managing predictions and evaluating and comparing different…”
BackgroundMethodTools for the automatic identification and classification of RNA base pairs
Yang, H.-W., Jossinet, F., Leontis, N., Chen, L., Westbrook, J., Berman, H., & Westhof, E. (2003) · Nucleic Acids Res. · doi:10.1093/nar/gkg529 · Cited by 331 · see citing works
FR3D: finding local and composite recurrent structural motifs in RNA 3D structures
Sarver, M., Zirbel, C. L., Stombaugh, J., Mokdad, A., & Leontis, N. (2007) · Journal of Mathematical Biology · doi:10.1007/s00285-007-0110-x · Cited by 310 · cited 1×
“This is done using 2D structure annotators such as RNAView, 21 MC-Annotate, 22 ClaRNA 23 or FR3D, 24 which provide base pairs and their classification.”
MethodD3R grand challenge 2015: evaluation of protein-ligand pose and affinity predictions.
Gathiaka, S., Liu, S., Chiu, M., Yang, H., Stuckey, J. A., Kang, Y. N., Delproposto, J., Kubish, G., Dunbar, J. B., Carlson, H. A., et al. (2016) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-016-9946-8 · Cited by 207 · cited 1× · see citing works
“Previous ligand docking challenges like Teach Discover Treat (TDT), 39 Continuous Evaluation of Ligand Prediction Performance (CELPP) 40 and Drug Discovery Data Resource (D3R) 34 – 37 have used two main types of metrics to assess how well participants can model receptor – ligand complexes.”
MethodNew metrics for comparing and assessing discrepancies between RNA 3D structures and models.
Parisien, M., Cruz, J. A., Westhof, E., & Major, F. (2009) · RNA: A publication of the RNA Society · doi:10.1261/rna.1700409 · Cited by 204 · cited 1× · see citing works
“We additionally consider an RNA-specific measure, Interaction Network Fidelity (INF), 13,20 which evaluates different types of RNA interactions in models.”
MethodQuantitative analysis of nucleic acid three-dimensional structures
Gendron, P., Lemieux, S., & Major, F. (2001) · Journal of Molecular Biology · doi:10.1006/jmbi.2001.4626 · Cited by 195 · cited 1× · see citing works
“22 Further, we will work to improve measures comparing predictions to raw data to assess low-resolution targets.”
Gaieb, Z., Liu, S., Gathiaka, S., Chiu, M., Yang, H., Shao, C., Feher, V. A., Walters, W. P., Kuhn, B., Rudolph, M. G., et al. (2018) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-017-0088-4 · Cited by 193 · see citing works
RNA-Puzzles: a CASP-like evaluation of RNA three-dimensional structure prediction.
Cruz, J. A., Blanchet, M.-F., Boniecki, M. J., Bujnicki, J. M., Chen, S.-J., Cao, S., Das, R., Ding, F., Dokholyan, N., Flores, S., et al. (2012) · RNA: A publication of the RNA Society · doi:10.1261/rna.031054.111 · Cited by 157 · see citing works
Self-citationCAD‐score: A new contact area difference‐based function for evaluation of protein structural models
Olechnovič, K., Kulberkytė, E., & Venclovas, Č. (2013) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.24172 · Cited by 153 · see citing works
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26617 · Cited by 148 · see citing works
Self-citationAssessment of three‐dimensional RNA structure prediction in CASP15
Das, R., Kretsch, R. C., Simpkin, A. J., Mulvaney, T., Pham, P., Rangan, R., Bu, F., Keegan, R. M., Topf, M., Rigden, D. J., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26602 · Cited by 139 · cited 1× · see citing works
“In 5 – 8 The main goal of this paper is to address technical issues of introducing the new categories in CASP and serve as a point of reference for format specifications and details on how the experiment was conducted.”
BackgroundSelf-citationD3R Grand Challenge 3: blind prediction of protein–ligand poses and affinity rankings
Gaieb, Z., Parks, C. D., Chiu, M. W., Yang, H., Shao, C., Walters, W. P., Lambert, M. H., Nevins, N., Bembenek, S. D., Ameriks, M. K., et al. (2019) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-018-0180-4 · Cited by 137 · see citing works
Parks, C. D., Gaieb, Z., Chiu, M. W., Yang, H., Shao, C., Walters, W. P., Jansen, J. M., McGaughey, G. B., Lewis, R. A., Bembenek, S. D., et al. (2020) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-020-00289-y · Cited by 137 · cited 1× · see citing works
“Previous ligand docking challenges like Teach Discover Treat (TDT), 39 Continuous Evaluation of Ligand Prediction Performance (CELPP) 40 and Drug Discovery Data Resource (D3R) 34 – 37 have used two main types of metrics to assess how well participants can model receptor – ligand complexes.”
MethodAssessment of the assessment: Evaluation of the model quality estimates in CASP10
Kryshtafovych, A., Barbato, A., Fidelis, K. A., Monastyrskyy, B., Schwede, T. F., & Tramontano, A. L. (2013) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.24347 · Cited by 131 · see citing works
Self-citationAssessment of predictions in the model quality assessment category
Cozzetto, D., Kryshtafovych, A., Ceriani, M., & Tramontano, A. (2007) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21669 · Cited by 130 · cited 1× · see citing works
“32 Given the recent advances in the accuracy of protein modeling methods, 11,12 CASP organizers decided to include prediction of protein-and RNA-small molecule ligand complexes into the scope of CASP15 experiment hoping to boost development of methods in this area.”
MethodSelf-citationEvaluation of model quality predictions in CASP9
Kryshtafovych, A., Fidelis, K., & Tramontano, A. (2011) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.23180 · Cited by 90 · see citing works
Self-citationRNA-align: quick and accurate alignment of RNA 3D structures based on size-independent TM-scoreRNA
Gong, S., Zhang, C.-X., & Zhang, Y. (2019) · Bioinform. · doi:10.1093/bioinformatics/btz282 · Cited by 89 · cited 1× · see citing works
“Evaluation measures include Clashscore, 15 Root Mean Square Deviation (RMSD), 16 Local Distance Difference Test (lDDT), 17 Template Modeling score (TM-score), 18 and Global Distance Test-Total Score (GDT-TS).”
MethodAssessment of ligand binding residue predictions in CASP9
Schmidt, T., Haas, J., Cassarino, T. G., & Schwede, T. (2011) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.23174 · Cited by 83 · see citing works
Self-citationMagnus, M., Antczak, M., Zok, T., Wiedemann, J., Lukasiak, P., Cao, Y., Bujnicki, J., Westhof, E., Szachniuk, M., & Miao, Z.-C. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz1108 · Cited by 78 · cited 1× · see citing works
“We additionally consider an RNA-specific measure, Interaction Network Fidelity (INF), 13,20 which evaluates different types of RNA interactions in models.”
MethodSelf-citationKryshtafovych, A., Barbato, A., Monastyrskyy, B., Fidelis, K., Schwede, T., & Tramontano, A. (2015) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.24919 · Cited by 77 · see citing works
Self-citationEvaluation of CASP8 model quality predictions
Cozzetto, D., Kryshtafovych, A., & Tramontano, A. L. (2009) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.22534 · Cited by 76 · see citing works
Self-citationAssessment of predictions submitted for the CASP7 function prediction category
López, G., Rojas, A., Tress, M., & Valencia, A. (2007) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21651 · Cited by 67 · see citing works
Assessment of model accuracy estimations in CASP12
Kryshtafovych, A., Monastyrskyy, B., Fidelis, K., Schwede, T., & Tramontano, A. (2017) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25371 · Cited by 63 · see citing works
Self-citationWon, J., Baek, M., Monastyrskyy, B., Kryshtafovych, A., & Seok, C. (2019) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25804 · Cited by 57 · see citing works
Self-citationAssessment of protein model structure accuracy estimation in CASP14 : Old and new challenges
Kwon, S., Won, J., Kryshtafovych, A., & Seok, C. (2021) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26192 · Cited by 56 · cited 2× · see citing works
“48 It has attracted the attention of many developers, with over 70 methods tested in the previous CASP experiment.”
BackgroundMethodSelf-citationWaleń, T., Chojnowski, G., Gierski, P., & Bujnicki, J. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku765 · Cited by 50 · cited 1× · see citing works
“This is done using 2D structure annotators such as RNAView, 21 MC-Annotate, 22 ClaRNA 23 or FR3D, 24 which provide base pairs and their classification.”
MethodAssessment of protein–ligand complexes in CASP15
Robin, X., Studer, G., Durairaj, J., Eberhardt, J., Schwede, T., & Walters, W. P. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26601 · Cited by 47 · see citing works
Self-citationContinuous Evaluation of Ligand Protein Predictions: A Weekly Community Challenge for Drug Docking
Wagner, J., Churas, C., Liu, S., Swift, R. V., Chiu, M., Shao, C., Feher, V., Burley, S., Gilson, M., & Amaro, R. (2018) · bioRxiv · doi:10.1101/469940 · Cited by 45 · cited 1×
“Previous ligand docking challenges like Teach Discover Treat (TDT), 39 Continuous Evaluation of Ligand Prediction Performance (CELPP) 40 and Drug Discovery Data Resource (D3R) 34 – 37 have used two main types of metrics to assess how well participants can model receptor – ligand complexes.”
MethodAssessment of ligand binding site predictions in CASP10
Cassarino, T. G., Bordoli, L., & Schwede, T. (2014) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.24495 · Cited by 42 · cited 1× · see citing works
“32 Given the recent advances in the accuracy of protein modeling methods, 11,12 CASP organizers decided to include prediction of protein-and RNA-small molecule ligand complexes into the scope of CASP15 experiment hoping to boost development of methods in this area.”
MethodSelf-citationThe prediction of protein function at CASP6
Soro, S., & Tramontano, A. (2005) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.20738 · Cited by 38
Evaluation of the stereochemical quality of predicted RNA 3D models in the RNA-Puzzles submissions
Carrascoza, F., Antczak, M., Miao, Z., Westhof, É., & Szachniuk, M. (2021) · RNA · doi:10.1261/rna.078685.121 · Cited by 32 · cited 1× · see citing works
“10,11 In CASP15, the focus of the EMA category shifted from predicting accuracy of single-sequence proteins to multi-molecular complexes.”
BackgroundSelf-citationBreaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15
Kryshtafovych, A., Montelione, G., Rigden, D., Mesdaghi, S., Karaca, E., & Moult, J. (2023) · doi:10.22541/au.169199504.48580490/v1 · Cited by 31 · see citing works
Self-citationAssessment of the assessment—All about complexes
Studer, G., Tauriello, G., & Schwede, T. F. (2023) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26612 · Cited by 31 · cited 2× · see citing works
“Finally, a validation report is created showing the results of the validation process to aid in troubleshooting invalid submissions.”
BackgroundMethodSelf-citationCragnolini, T., Sahota, H., Joseph, A., Sweeney, A., Malhotra, S., Vasishtan, D., & Topf, M. (2021) · Acta Crystallographica Section D: Structural Biology · doi:10.1107/s2059798320014928 · Cited by 31 · cited 1× · see citing works
“25 Given two sets of interactions, one for the model and 27,28 It is worth noting that model-independent measures are known to overinflate scores for compressed models with unrealistic geometries.”
BackgroundSelf-citationBreaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15
Kryshtafovych, A., Montelione, G. T., Rigden, D. J., Mesdaghi, S., Karaca, E., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26584 · Cited by 31 · see citing works
Self-citationTeach-Discover-Treat (TDT): Collaborative Computational Drug Discovery for Neglected Diseases
Jansen, J., Cornell, W. D., Tseng, Y., & Amaro, R. (2012) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2012.07.007 · Cited by 16 · cited 1× · see citing works
“Previous ligand docking challenges like Teach Discover Treat (TDT), 39 Continuous Evaluation of Ligand Prediction Performance (CELPP) 40 and Drug Discovery Data Resource (D3R) 34 – 37 have used two main types of metrics to assess how well participants can model receptor – ligand complexes.”
Methodrna-tools.online: a Swiss army knife for RNA 3D structure modeling workflow
Magnus, M. (2022) · Nucleic Acids Res. · doi:10.1093/nar/gkac372 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Simulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937 · see citing works
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26617 · Cited by 148 · see citing works
“17 The first is for modeling multiple structures in ensembles, both protein and RNA.”
BackgroundSelf-citationAssessment of three‐dimensional RNA structure prediction in CASP15
Das, R., Kretsch, R. C., Simpkin, A. J., Mulvaney, T., Pham, P., Rangan, R., Bu, F., Keegan, R. M., Topf, M., Rigden, D. J., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26602 · Cited by 139 · see citing works
Self-citationThe structure assessment web server: for proteins, complexes and more
Waterhouse, A. M., Studer, G., Robin, X., Bienert, S., Tauriello, G., & Schwede, T. F. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae270 · Cited by 140 · see citing works
Self-citationLi, Y., Zhang, C.-X., Feng, C.-J., Pearce, R., Freddolino, P., & Zhang, Y. (2022) · bioRxiv · doi:10.1038/s41467-023-41303-9 · Cited by 114 · see citing works
Current and future directions in network biology
Žitnik, M., Li, M. M., Wells, A., Glass, K., Gysi, D. M., Krishnan, A., Murali, T. M., Predrag Radivojac, A., Roy, S., Baudot, A., et al. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae099 · Cited by 118 · see citing works
When will RNA get its AlphaFold moment?
Schneider, B., Sweeney, B., Bateman, A. G., Černý, J., Żok, T., & Szachniuk, M. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad726 · Cited by 113 · see citing works
Self-citationHave protein-ligand cofolding methods moved beyond memorisation?Preprint
Škrinjar, P., Eberhardt, J., Tauriello, G., Schwede, T., & Durairaj, J. (2025) · bioRxiv · doi:10.1101/2025.02.03.636309 · Cited by 100 · see citing works
“Recently, CASP15 had a pilot experiment in the evaluation of protein-ligand complex prediction [8], inspired by previous efforts such as Teach Discover Treat (TDT) [9], Continuous Evaluation of Ligand Prediction Performance (CELPP) [10], Drug Discovery Data Resource (D3R) [11–14], and Community…”
Self-citationArnold, A., McLellan, S., & Stokes, J. M. (2025) · npj Antimicrobials and Resistance · doi:10.1038/s44259-025-00085-4 · Cited by 84 · see citing works
“BOLTZ-1 was benchmarked against CHAI-1, another AlphaFold3 replication of equal performance 95,97,102 .”
Ozden, B., Kryshtafovych, A., & Karaca, E. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26598 · Cited by 51 · see citing works
“25,85,86 Figure 4A also suggests that the size of the complex is another factor influencing modeling accuracy, as larger targets (1800 residues or more) were modeled slightly worse than the smaller ones (average ICS of 0.68 vs. 0.75 for the latter).”
BackgroundInfluentialSelf-citationNithin, C., Kmiecik, S., Błaszczyk, R., Nowicka, J., & Tuszyńska, I. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae541 · Cited by 42 · see citing works
Szikszai, M., Magnus, M., Sanghi, S., Kadyan, S., Bouatta, N., & Rivas, E. (2024) · Journal of Molecular Biology · doi:10.1016/j.jmb.2024.168552 · Cited by 38 · see citing works
Szikszai, M., Magnus, M., Sanghi, S., Kadyan, S., Bouatta, N., & Rivas, E. (2024) · doi:10.1101/2024.01.30.578025 · Cited by 38 · see citing works
“…interest in RNA, partly attributed to AlphaFold’s success, and partly due to the rise of RNA-based therapeutics [19], CASP15 joined RNA-Puzzles [20] and added 12 RNA-only targets to the competition [21, 22], where all deep learning methods performed comparatively poorly to traditional tools [23].”
BackgroundMethodRNA target highlights in CASP15: Evaluation of predicted models by structure providers
Kretsch, R., Andersen, E., Bujnicki, J., Chiu, W., Das, R., Luo, B., Masquida, B., McRae, E., Schroeder, G. M., Su, Z.-M., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26550 · Cited by 38 · see citing works
Self-citationRNA 3D Structure Prediction: Progress and Perspective
Wang, X., Yu, S., Lou, E., Tan, Y.-L., & Tan, Z.-J. (2023) · Molecules · doi:10.3390/molecules28145532 · Cited by 43 · see citing works
“Here, RNA-Puzzles is a CASP-like competition for RNA 3D structure prediction [19–23], and CASP-RNA is a prediction competition for RNA 3D structures newly present in CASP15 [24].”
BackgroundInfluentialBreaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15
Kryshtafovych, A., Montelione, G., Rigden, D., Mesdaghi, S., Karaca, E., & Moult, J. (2023) · doi:10.22541/au.169199504.48580490/v1 · Cited by 31 · see citing works
Self-citationBreaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15
Kryshtafovych, A., Montelione, G. T., Rigden, D. J., Mesdaghi, S., Karaca, E., & Moult, J. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26584 · Cited by 31 · see citing works
“To build on these results, the scope of CASP15 (2022) was expanded to include several new areas where the deep learning methods hold promise for further progress (4).”
BackgroundSelf-citationAssessment of Pharmaceutical Protein–Ligand Pose and Affinity Predictions in CASP16
Gilson, M. K., Eberhardt, J., Škrinjar, P., Durairaj, J., Robin, X., & Kryshtafovych, A. (2025) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.70061 · Cited by 18 · see citing works
“In particular, CASP 15 (2022) was the first cycle that challenged participants to predict the three-dimensional structures of protein–small molecule (also called protein–ligand) complexes [7, 8].”
Self-citationIntegrating AlphaFold and deep learning for atomistic interpretation of cryo-EM maps
Dai, X., Wu, L., Yoo, S., & Liu, Q. (2023) · bioRxiv · doi:10.1101/2023.02.02.526877 · Cited by 23 · see citing works
Can Protein Structure Prediction Methods Capture Alternative Conformations of Membrane Transporters?
Xie, T., & Huang, J. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c01936 · Cited by 23 · see citing works
Can Protein Structure Prediction Methods Capture Alternative Conformations of Membrane Proteins?
Xie, T., & Huang, J. (2023) · doi:10.1101/2023.08.04.552045 · Cited by 23 · see citing works
“The Critical Assessment of protein Structure Prediction (CASP) sought to provide targets for predicting “protein conformational ensembles” in CASP15 (2022), which highlighted the importance of evaluating methods for predicting alternative conformations and structural ensembles [49].”
MethodEvaluating generalization in protein–ligand cofolding methods
Škrinjar, P., Eberhardt, J., Studer, G., Tauriello, G., Schwede, T., & Durairaj, J. (2026) · Nature Structural & Molecular Biology · doi:10.1038/s41594-026-01797-5 · Cited by 22 · see citing works
“Recently, CASP15 had a pilot experiment in the evaluation of protein–ligand complex prediction 8 , inspired by previous efforts such as Teach Discover Treat 9 , Continuous Evaluation of Ligand Prediction Performance 10 , Drug Discovery Data Resource 11–14 and Community Structure–Activity Resource…”
Self-citationAutomated benchmarking of combined protein structure and ligand conformation prediction
Leemann, M., Sagasta, A., Eberhardt, J., Schwede, T. F., Robin, X., & Durairaj, J. (2023) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26605 · Cited by 23 · see citing works
“10 These challenges include: (1) PLC with incomplete ligands or suboptimal quality to be used as ground truth ligand poses, (2) the need for extensive manual verification of data input and prediction output, and (3) the lack of suitable scoring metrics that consider both protein structure and…”
BackgroundSelf-citationRNAbpFlow: Base pair-augmented SE(3)-flow matching for conditional RNA 3D structure generation
Tarafder, S., & Bhattacharya, D. (2025) · bioRxiv · doi:10.1101/2025.01.24.634669 · Cited by 18
“Furthermore, many existing methods fail to fully leverage the RNA base pair (2D) information, including canonical and non-canonical base pairing interactions, key determinants of the final 3D conformation of RNA [24, 25].”
An outlook on structural biology after A lpha F old: tools, limits and perspectives
Rosignoli, S., Pacelli, M., Manganiello, F., & Paiardini, A. (2024) · FEBS Open Bio · doi:10.1002/2211-5463.13902 · Cited by 18 · see citing works
Recent advances and challenges in protein complex model accuracy estimation
Liang, F., Sun, M., Xie, L., Zhao, X., Liu, D., Zhao, K., & Zhang, G. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.04.049 · Cited by 15 · see citing works
“Based on the analysis of the CASP15 blind test results, we can conclude that the consensus methods perform well on most protein targets, but their performance significantly decreases on targets with a poor model pool [89].”
BackgroundMethodKnotted artifacts in predicted 3D RNA structures
Greń, B., Antczak, M., Zok, T., Sulkowska, J. I., & Szachniuk, M. (2024) · bioRxiv · doi:10.1101/2024.03.04.583268 · Cited by 13
“The CASP15 resources are available at https://predictioncenter.org/download_area/CASP15/predictions/RNA/ [17]).”
Self-citationDRfold2 is a deep learning-based tool that enables efficient and accurate RNA structure prediction
Li, Y., Feng, C., Zhang, X., Tsukiyama, S., Feng, D., & Zhang, Y. (2026) · PLoS Biology · doi:10.1371/journal.pbio.3003659 · Cited by 8 · see citing works
Systematic benchmarking of deep-learning methods for tertiary RNA structure prediction
Bahai, A., Kwoh, C. K., Mu, Y., & Li, Y. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1012715 · Cited by 12 · see citing works
lociPARSE: A Locality-aware Invariant Point Attention Model for Scoring RNA 3D Structures
Tarafder, S., & Bhattacharya, D. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01621 · Cited by 11 · see citing works
A fully automated benchmarking suite to compare macromolecular complexes
Studer, G., Robin, X., Bienert, S., Durairaj, J., Škrinjar, P., Tauriello, G., Waterhouse, A. M., & Schwede, T. (2026) · Nature Methods · doi:10.1038/s41592-025-02973-z · Cited by 10 · see citing works
“A variety of ligand pose prediction challenges have taken place, such as D3R 7–10 , CELPP 11 and others 12–16 , and ligand assessment has been included in CASP since 2022 17 .”
Self-citationKagaya, Y., Nakamura, T., Verburgt, J. C., Jain, A., Terashi, G., Punuru, P., Tugolukova, E., Park, J. H., Saha, A., Huang, D. Y. C., et al. (2025) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.70033 · Cited by 8 · see citing works
“The RNA structure prediction category was introduced in CASP15 for the first time [40, 41].”
The Need for Continuing Blinded Pose- and Activity Prediction Benchmarks
Krämer, C., Chodera, J. D., Damm‐Ganamet, K. L., Gilson, M. K., Günther, J., Lessel, U. F., Lewis, R. A., Mobley, D. L., Nittinger, E., Pecina, A., et al. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c02296 · Cited by 13 · see citing works
The landscape of RNA 3D structure modeling with transformer networks
Tarafder, S., Roche, R., & Bhattacharya, D. (2024) · Biology Methods and Protocols · doi:10.1093/biomethods/bpae047 · Cited by 10 · see citing works
“…discrepancy can be the presence of a higher number of noncanonical base pairings in larger RNAs, such as ribosomal RNAs (rRNAs) and long non-coding RNAs (lncRNAs); and accurate RNA 3D structure prediction relies heavily on correctly predicting both noncanonical and canonical base pairings [16].”
Wang, B., & Li, W.-J. (2024) · Genes · doi:10.3390/genes15081090 · Cited by 9 · see citing works
“In CASP 15 [36], which focuses more on protein complex and RNA structure prediction, Alchemy RNA learns richer sequence information through pre-trained RNA language models (RNA-FM [37]), ranking first among all AI methods.”
BackgroundAb initio RNA structure prediction with composite language model and denoised end-to-end learningPreprint
Li, Y., Feng, C., Zhang, X., & Zhang, Y. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.03.05.641632 · Cited by 7 · see citing works
Li, J., Zhu, Z., & Song, C. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.09.598121 · Cited by 6 · see citing works
“14,15 In CASP15, prediction targets encompassed ensembles of protein conformations, 16 and there is growing interest in developing computational approaches to predict alternative conformations in the field.”
BackgroundZEPPI: proteome-scale sequence-based evaluation of protein-protein interaction modelsPreprint
Zhao, H., Murray, D., Petrey, D. S., & Honig, B. (2023) · Research Square · doi:10.21203/rs.3.rs-3289791/v1 · Cited by 7 · see citing works
“There have been continuing improvements in AFM-based methods 13,14 as is evident from the substantial progress in the recent CASP-CAPRI 15,16 experiment 17 .”
MethodRNAGym: Large-scale Benchmarks for RNA Fitness and Structure PredictionPreprint
Arora, R., Angelo, M., Choe, C. A., Shearer, C. A., Kollasch, A. W., Qu, F., Weitzman, R., Gazizov, A., Gurev, S. F., Xie, E., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.16.660049 · Cited by 7 · see citing works
“CASP15 [Kryshtafovych et al., 2023], the latest iteration of CASP, introduced a dedicated Figure 1: RNAGym benchmarks.”
Robin, X., Škrinjar, P., Waterhouse, A., Studer, G., Tauriello, G., Durairaj, J., & Schwede, T. (2025) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.70060 · Cited by 5 · see citing works
Self-citationUpdates to the CASP Infrastructure in 2024
Kryshtafovych, A., Miłostan, M., Lensink, M. F., Velankar, S. S., Bonvin, A. M. J. J., Moult, J., & Fidelis, K. A. (2025) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.70042 · Cited by 5 · see citing works
“The previous paper in this series [10] provided a detailed report on model formats, evaluation measures, and general principles for handling the three CASP15 pilot categories: RNA structure, ligand–protein complexes, and ensembles of alternative conformations.”
InfluentialSelf-citationToward high-resolution modeling of small molecule–ion channel interactions
Harris, B. J., Nguyen, P., Zhou, G., Wulff, H., Dimaio, F., & Yarov-Yarovoy, V. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2024.1411428 · Cited by 5 · see citing works
ZEPPI: Proteome-scale sequence-based evaluation of protein–protein interaction models
Zhao, H., Petrey, D. S., Murray, D., & Honig, B. H. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2400260121 · Cited by 12 · see citing works
“There have been continuing improvements in AFM - based methods (13, 14) as is evident from the substantial progress in the recent CASP - CAPRI (15, 16) experiment (17).”
RNAtive to recognize native-like structure in a set of RNA 3D models
Pielesiak, J., Antczak, M., Szachniuk, M., & Zok, T. (2025) · bioRxiv · doi:10.1093/bioinformatics/btaf601 · Cited by 4 · see citing works
“In the benchmark, we used two RNA 3D structure data-sets: CASP15 (Kryshtafovych et al. 2023; Das et al. 2023) and the randstr decoy set (Capriotti et al. 2011).”
Self-citationDatabases and web-based tools for studying structures of protein-nucleic acid complexes.
Dapkūnas, J., & Venclovas, Č. (2025) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2025.103079 · Cited by 4 · see citing works
Comprehensive datasets for RNA design, machine learning, and beyond
Badura, J., Rybarczyk, A., & Zok, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-07041-2 · Cited by 3 · see citing works
Self-citationRNAbpFlow: base pair-augmented SE(3) flow matching for conditional RNA 3D structure generation
Tarafder, S., & Bhattacharya, D. (2026) · Nature Methods · doi:10.1038/s41592-026-03128-4 · Cited by 3
“Furthermore, many existing methods fail to fully leverage the RNA base pair (2D) information, including canonical and noncanonical base-pairing interactions, key determinants of the final 3D conformation of RNA 25,26 .”
Liu, G., Mu, K., Ran, F., Liu, J., Zhou, L., Peng, L., Guo, F., Liu, Y., Wei, F.-D., Zhu, L.-L., et al. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e38203 · Cited by 3 · see citing works
Protein quality assessment with a loss function designed for high-quality decoys
Roy, S., & Ben‐Hur, A. (2023) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2023.1198218 · Cited by 3 · see citing works
“In CASP15, the focus shifted from predicting the accuracy of single-chain decoys to that of multi-chain complexes (Kryshtafovych et al., 2023).”
BackgroundZhang, L., Ding, R., Chen, X., Hou, J., Si, D., Wang, Y., Lin, K.-Y., & Cao, R.-Z. (2025) · bioRxiv · doi:10.1002/pro.70734 · Cited by 1 · see citing works
PSBench: a large-scale benchmark for estimating the accuracy of protein complex structural models
Neupane, P., Liu, J., & Cheng, J.-L. (2025) · Neural Information Processing Systems · doi:10.48550/arxiv.2505.22674 · Cited by 1
“To stimulate the development of EMA methods for protein complex structural models, since 2020, CASP has dedicated one competition category to assess EMA methods[13, 14].”
Qiu, Z.-J., Xu, S., Wei, J., Shen, T., & Sun, S. (2024) · bioRxiv · doi:10.1101/2024.05.27.595311 · Cited by 1
Benchmarking protein-ligand co-folding across Nav, Cav and Kv-family channels
Zhu, Y., & Rahman, T. (2026) · Frontiers in Biophysics · doi:10.3389/frbis.2026.1937302
Recent Advances in Estimation of Protein Structure Model Accuracy
Liu, D., Cui, X.-Y., Wang, H.-D., & Zhang, G.-J. (2023) · Acta Physica Sinica · doi:10.7498/aps.72.20231071
- Authors
- Kryshtafovych, A., Antczak, M., Szachniuk, M., Zok, T., Kretsch, R. C., Rangan, R., Pham, P., Das, R., Robin, X., Studer, G., Durairaj, J., Eberhardt, J., Sweeney, A., Topf, M., Schwede, T., Fidelis, K., & Moult, J.
- Affiliations
- University of California, Davis (US)
- Institute of Bioorganic Chemistry, Polish Academy of Sciences (PL)
- Institute of Computing Science, Poznan University of Technology Poznan Poland
- Stanford Medicine (US)
- Howard Hughes Medical Institute (US)
- SIB Swiss Institute of Bioinformatics (CH)
- Biozentrum, University of Basel Basel Switzerland
- Leibniz Institute of Virology (LIV) (DE)
- Institute for Bioscience and Biotechnology Research (US)
- Corresponding author
- Kryshtafovych, Andriy
- ORCID
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2023-06-12
- Volume
- 91
- Issue
- 12
- Pages
- 1550-1557
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- PMC
- PMC10713864
- OpenAlex
- W4380291688
- Cited by
- 54
- References
- 52
- Citations by year
- 2026: 4 · 2025: 16 · 2024: 19 · 2023: 15
- FWCI
- 4.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proteins Structure Function and Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- edoc (University of Basel) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Funders
- Howard Hughes Medical Institute
- National Institutes of Health (R35 GM122579)
- Wellcome Trust (209250/Z/17/Z)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Keywords
- CASP15, protein structure prediction, RNA structure prediction, ligand-protein complexes, alternative conformations, ensembles, data management systems
- MeSH terms
- Ligands, Models, Molecular, Protein Conformation, Proteins, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
