Assessing scoring metrics for AlphaFold2 and AlphaFold3 protein complex predictions
Open accessPublished version of a preprint
Abstract
Recent breakthroughs in AI‐driven protein structure prediction have revolutionized structural biology, unlocking new possibilities to model complex biomolecular interactions. We evaluated widely used scoring metrics for assessing models predicted by ColabFold with templates, ColabFold without templates, and AlphaFold3. We benchmarked the optimal cutoffs for these assessment scores using a set of 223 heterodimeric, high‐resolution protein structures and their predictions. Our results show that ColabFold with templates and AlphaFold3 perform similarly, and both outperform ColabFold without templates. However, the assessment scores perform best on ColabFold without templates. Furthermore, interface‐specific scores are more reliable for evaluating protein complex predictions compared to the corresponding global scores. Notably, ipTM and model confidence achieve the best discrimination between correct and incorrect predictions. Based on our results, we developed a weighted combined score, C2Qscore, to improve model quality assessment. We used C2Qscore to analyze dimers from large assemblies solved by cryoEM, revealing potential limitations of the existing metrics when multiple configurations of heterodimers are possible. This study provides insights into the strengths and weaknesses of current scores and offers guidance for improving protein complex model assessment under realistic use case conditions. C2Qscore has been integrated as a tool into our ChimeraX plug‐in PICKLUSTER v.2.0 and is also available as a command‐line tool on https://gitlab.com/topf-lab/c2qscore .
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · see citing works
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024) · Nature · doi:10.1038/s41586-024-07487-w · Cited by 15,861 · see citing works
Cd-hit: Accelerated for clustering the next-generation sequencing data
Fu, L., Niu, B., Zhu, Z., Wu, S., & Li, W. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts565 · Cited by 11,175 · see citing works
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391 · see citing works
UCSF ChimeraX : Structure visualization for researchers, educators, and developers
Pettersen, E. F., Goddard, T. D., Huang, C. C., Meng, E. C., Couch, G. S., Croll, T. I., Morris, J. H., & Ferrin, T. E. (2020) · Protein Science · doi:10.1002/pro.3943 · Cited by 9,327 · see citing works
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539 · see citing works
Comparison of multiple Amber force fields and development of improved protein backbone parameters
Horn̆ák, V., Abel, R., Okur, A., Strockbine, B., Roitberg, A. E., & Simmerling, C. L. (2006) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.21123 · Cited by 6,903 · see citing works
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440 · see citing works
MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets
Steinegger, M., & Söding, J. (2017) · Nature Biotechnology · doi:10.1038/nbt.3988 · Cited by 4,840 · see citing works
Protein complex prediction with AlphaFold-Multimer
Evans, R., O’Neill, M., Pritzel, A., Antropova, N., Senior, A., Green, T., Žídek, A., Bates, R., Blackwell, S., Yim, J., et al. (2021) · doi:10.1101/2021.10.04.463034 · Cited by 3,808 · see citing works
Protein Structure Prediction and Structural Genomics
Baker, D., & Sali, A. (2001) · Science · doi:10.1126/science.1065659 · Cited by 1,587 · see citing works
Noise Reduction in Speech Processing
Cohen, I., Huang, Y. A., Chen, J., & Benesty, J. (2009) · Springer topics in signal processing · doi:10.1007/978-3-642-00296-0 · Cited by 1,466 · see citing works
Improved prediction of protein-protein interactions using AlphaFold2
Bryant, P., Pozzati, G., & Elofsson, A. (2021) · doi:10.1101/2021.09.15.460468 · Cited by 1,025 · see citing works
SAbDab: the structural antibody database
Dunbar, J., Krawczyk, K., Leem, J., Baker, T., Fuchs, A., Georges, G., Shi, J., & Deane, C. M. (2014) · Nucleic Acids Research · doi:10.1093/nar/gkt1043 · Cited by 674 · see citing works
Accelerating Protein Docking in ZDOCK Using an Advanced 3D Convolution Library
Pierce, B. G., Hourai, Y., & Weng, Z. (2011) · PLoS ONE · doi:10.1371/journal.pone.0024657 · Cited by 645 · see citing works
DockQ: A Quality Measure for Protein-Protein Docking Models
Basu, S., & Wallner, B. (2016) · PLOS ONE · doi:10.1371/journal.pone.0161879 · Cited by 460 · see citing works
Cavanaugh, J. (2006) · Journal of the American Statistical Association · doi:10.1198/jasa.2006.s94 · Cited by 436 · see citing works
Benchmarking AlphaFold for protein complex modeling reveals accuracy determinants
Yin, R., Feng, B. Y., Varshney, A., & Pierce, B. G. (2021) · doi:10.1101/2021.10.23.465575 · Cited by 335 · see citing works
Benchmarking <scp>AlphaFold</scp> for protein complex modeling reveals accuracy determinants
Yin, R., Feng, B. Y., Varshney, A., & Pierce, B. G. (2022) · Protein Science · doi:10.1002/pro.4379 · Cited by 335 · see citing works
Smyth, M., & Martin, J. H. J. (2000) · Molecular Pathology · doi:10.1136/mp.53.1.8 · Cited by 327 · see citing works
Docking, scoring, and affinity prediction in CAPRI
Lensink, M. F., & Wodak, S. J. (2013) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.24428 · Cited by 249 · see citing works
Principles of Protein Structure
Harrison, P. (1980) · FEBS Letters · doi:10.1016/0014-5793(80)81239-7 · Cited by 240 · see citing works
Evaluation of AlphaFold-Multimer prediction on multi-chain protein complexes
Zhu, W., Shenoy, A., Kundrotas, P., & Elofsson, A. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad424 · Cited by 202 · see citing works
Computational prediction of protein interfaces: A review of data driven methods
Xue, L. C., Dobbs, D. L., Bonvin, A. M. J. J., & Honavar, V. (2015) · FEBS Letters · doi:10.1016/j.febslet.2015.10.003 · Cited by 184 · see citing works
Assessment of the assessment: Evaluation of the model quality estimates in CASP10
Kryshtafovych, A., Barbato, A., Fidelis, K. A., Monastyrskyy, B., Schwede, T. F., & Tramontano, A. L. (2013) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.24347 · Cited by 131 · see citing works
Assessment of predictions in the model quality assessment category
Cozzetto, D., Kryshtafovych, A., Ceriani, M., & Tramontano, A. (2007) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.21669 · Cited by 130 · see citing works
The scoring of poses in protein-protein docking: current capabilities and future directions
Moal, I. H., Torchala, M., Bates, P. A., & Fernández‐Recio, J. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-286 · Cited by 116 · see citing works
Mirabello, C., & Wallner, B. (2024) · bioRxiv · doi:10.1093/bioinformatics/btae586 · Cited by 110 · see citing works
Fold assessment for comparative protein structure modeling
Melo, F., & Šali, A. (2007) · Protein Science · doi:10.1110/ps.072895107 · Cited by 105 · see citing works
From interaction networks to interfaces, scanning intrinsically disordered regions using AlphaFold2
Bret, H., Gao, J., Zea, D. J., Andreani, J., & Guerois, R. (2024) · Nature Communications · doi:10.1038/s41467-023-44288-7 · Cited by 103 · see citing works
VoroMQA web server for assessing three-dimensional structures of proteins and protein complexes
Olechnovič, K., & Venclovas, Č. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz367 · Cited by 69 · see citing works
Olechnovič, K., & Venclovas, Č. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26554 · Cited by 38 · see citing works
Pándy‐Szekeres, G., Herrera, L. P. T., Caroli, J., Kermani, A. A., Kulkarni, Y., Keserű, G. M., & Gloriam, D. E. I. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkad1089 · Cited by 36 · see citing works
Assessment of the assessment—All about complexes
Studer, G., Tauriello, G., & Schwede, T. F. (2023) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26612 · Cited by 31 · see citing works
Local resolution estimates of cryoEM reconstructions
Vilas, J. L., Heymann, J. B., Tagare, H., Ramírez-Aportela, E., Carazo, J. M., & Sorzano, C. Ó. S. (2020) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2020.06.005 · Cited by 23 · see citing works
Estimation of model accuracy in CASP15 using the M odFOLDdock server
Edmunds, N. S., Alharbi, S. M. A., Genc, A. G., Adiyaman, R., & McGuffin, L. J. (2023) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.26532 · Cited by 23 · see citing works
Schweke, H., Xu, Q., Tauriello, G., Pantolini, L., Schwede, T. F., Cazals, F., Lhéritier, A., Fernández‐Recio, J., Rodríguez‐Lumbreras, L. A., Schueler‐Furman, O., et al. (2023) · PROTEOMICS · doi:10.1002/pmic.202200323 · Cited by 23 · see citing works
Recent progress and future perspective of electron cryomicroscopy for structural life sciences
Namba, K., & Makino, F. (2021) · Microscopy · doi:10.1093/jmicro/dfab049 · Cited by 18 · see citing works
PICKLUSTER: a protein-interface clustering and analysis plug-in for UCSF ChimeraX
Genz, L. R., Mulvaney, T., Nair, S., & Topf, M. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad629 · Cited by 14 · see citing works
Self-citationPPIscreenML: Structure-based screening for protein-protein interactions using AlphaFoldPreprint
Mischley, V., Maier, J., Chen, J., & Karanicolas, J. (2024) · eLife · doi:10.7554/elife.98179.1 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zhang, C., Wang, W., Zhu, N., Cao, Z., Wu, Y., Mao, Q., Zhu, C., Zhang, C., Guo, J., & Duan, H. (2025) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5c01393 · Cited by 13 · see citing works
Smorodina, E., Ali, M., Kropivšek, K., Salicari, L., Miklavc, S., Kappassov, A., Fu, C., Sormanni, P., de Marco, A., & Greiff, V. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.02.709004 · Cited by 12 · see citing works
Ünsal, S., Holland, B., Sardag, I., & Timuçin, E. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag137 · Cited by 9 · see citing works
Diversity and distribution of bacterial DNA polymerases
Timinskas, K., Kazlauskas, D., Timinskas, A., & Venclovas, Č. (2026) · Nucleic Acids Research · doi:10.1093/nar/gkag133 · Cited by 3 · see citing works
NanoporeDB: a structural resource of multimeric protein nanopores for single-molecule sensing
Liu, Y., Su, Z., Yang, W., Li, D., Zhang, J., Zhang, Y., Zeng, T., Zhang, Y., Li, Y., Fan, G., et al. (2026) · GigaScience · doi:10.1093/gigascience/giag076 · Cited by 2 · see citing works
Kotel'nikov, D. D., Tatarinova, K. S., & Zhdanov, D. D. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26199262 · Cited by 1 · see citing works
AlphaFold-Multimer modelling of linked nAChR subunits challenges concatemer design assumptions
Sahlström, H. M., Rufener, L., Horsberg, K. H., Sarr, A., Horsberg, T. E., & Bakke, M. J. (2026) · Scientific Reports · doi:10.1038/s41598-026-50253-3 · Cited by 1 · see citing works
A simple probabilistic AlphaFold interaction score
Badonyi, M., & Tóth-Petróczy, Á. (2026) · Protein Science · doi:10.1002/pro.70760 · Cited by 1 · see citing works
The huntingtin–HAP40 complex is a bidirectional cellular rheostatPreprint
Seefelder, M., Klein, F. A. C., Calzia, E., Muqaku, B., Oeckl, P., & Kochanek, S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.03.729778 · Cited by 1 · see citing works
Large-scale structure prediction of DUF-containing protein-protein interactionsPreprint
Riepenhausen, L., Costa, F., Andreeva, A. E., & Bateman, A. G. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.19.745780 · Cited by 1 · see citing works
Diaz, F. M., Milano, C. R., Huo, J., Hochwagen, A., & Corbett, K. D. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.01.19.700424
Tsai, H.-C. (2026) · Research Square · doi:10.21203/rs.3.rs-9526911/v1
Partner determination from protein sequences using class informationPreprint
Gennai, L., Caredda, F., Rebeaud, M. E., Pagnani, A., & Rios, P. D. L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.06.723283
Zhang, R., Zhu, X., Xue, Y. J., Tang, X., Yao, Y., Cao, D., He, W., Guo, C., Liu, X. Y., & Shu, Y. (2026) · Journal of Agriculture and Food Research · doi:10.1016/j.jafr.2026.103139
You, M., Li, H., Xie, Y., Liu, Z. L., Wei, X., Wu, J., Xu, Y., Zhang, Q., Zhang, J., Xu, Q., et al. (2026) · Frontiers in Plant Science · doi:10.3389/fpls.2026.1891189
Ensembles of in silico structures enable T cell peptide-MHC binding predictionPreprint
Lyudovyk, O., Levine, J., Pathil, M., Martis, S., Streltsov, A., Sethna, Z. M., Elhanati, Y., Balachandran, V., Morris, Q., & Greenbaum, B. D. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.14.738340
ProNA3D: Distance-Based Analysis of Nucleic Acid-Containing Interfaces
Genz, L. R., & Topf, M. (2026) · Computational and Structural Biotechnology Journal · doi:10.34133/csbj.0202
Self-citationStructural quality-tier assessment for TCR-pMHC functional enrichment
Ascunce-París, A., Romero‐Durana, M., Valencia, A., Farriol-Duran, R., & Guallar, V. (2026) · Frontiers in Immunology · doi:10.3389/fimmu.2026.1869810
Marien, J., Sritharan, S., Caviglia, B., Versini, R., Auclair, L., Barraud, P., Tisné, C., Reguei, A., Murail, S., Leclerc, F., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.24.740587
Zhang, L., Jiang, J., Yang, W., Mu, L., Han, L., Zhang, A., & Li, L. (2026) · Biotechnology Advances · doi:10.1016/j.biotechadv.2026.108988
Benchmarking confidence estimation and rescoring for cyclic peptide–protein complex predictionsPreprint
Li, Z., Yuan, Y., Hu, K., Pan, P., & He, F. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.20.746104
Lin, S.-R., Li, M., Wang, S., Li, E., Sun, H., & Li, L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.19.745820
Yu, K., Wang, Z., Chen, X., Lei, S., Feng, G., Zhang, W., Fu, F., Wang, A., Wang, Q., & Pei, X. (2026) · Bioorganic Chemistry · doi:10.1016/j.bioorg.2026.110591
Chernyshova, A. I., & Op, Z. (2026) · Journal of microbiology epidemiology immunobiology · doi:10.36233/0372-9311-881
Washim, M. R., Irishika, C., Harada, N., Kawaguchi, K., Yo, O., Kono, T., & Hikima, J.-I. (2026) · Comparative Immunology Reports · doi:10.1016/j.cirep.2026.200322
Challenges in AI-based structural modeling of host–pathogen protein interactions
Gilep, K., & Kosiński, J. (2026) · Current Opinion in Virology · doi:10.1016/j.coviro.2026.101587
- Authors
- Genz, L. R., Nair, S., Nagar, N., & Topf, M.
- Affiliations
- Leibniz Institute of Virology (LIV) (DE)
- Corresponding author
- Topf, Maya
- Venue
- Protein Science
- Publisher
- Wiley
- Published
- 2025-10-13
- Volume
- 34
- Issue
- 11
- Pages
- e70327-e70327
- ISSN
- 0961-8368 (ISSN-L)
- 1469-896X
- PMC
- PMC12516916
- OpenAlex
- W4415147097
- Cited by
- 17
- References
- 41
- Citations by year
- 2026: 17
- FWCI
- 5.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Protein Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (CRC1648)
- Bundesministerium für Bildung und Forschung (031L028)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Keywords
- protein complex structure prediction, scoring metrics, AlphaFold3, ColabFold, model confidence, template-based modeling, model quality assessment
- MeSH terms
- Models, Molecular, Protein Conformation, Proteins, Software, Protein Folding, Computational Biology, Databases, Protein, Protein Multimerization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
