Evaluating generalization in protein–ligand cofolding methods
Open accessPublished version of a preprint
Abstract
Deep learning has driven major breakthroughs in protein structure prediction; however, one of the next critical steps forward is accurately predicting how proteins interact with small-molecule ligands, to enable real-world applications such as drug discovery. Recent cofolding methods aim to address this challenge, but evaluating their performance has been inconclusive because of the lack of relevant benchmarking datasets. Here we present a comprehensive evaluation of four leading all-atom cofolding methods using our newly introduced benchmark dataset, Runs N’ Poses. Runs N’ Poses comprises 2,600 high-resolution protein–ligand systems released after the training cutoff used by these methods. We demonstrate that current cofolding approaches largely memorize ligand poses from their training data, hindering their use for de novo drug design. With this assessment and benchmark dataset, we aim to accelerate progress in the field by allowing for a more realistic assessment of the current state-of-the-art deep learning methods for predicting protein–ligand interactions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · cited 3× · see citing works
“Deep learning methods such as AlphaFold2 21 have revolutionized protein structure prediction, inspiring the development of all-atom cofolding methods like RoseTTAFold All-Atom 22 , AlphaFold3 23 , Chai-1 24 , Boltz-1 25 , Boltz-2 26 , Protenix 27 and NeuralPLexer3 28 .”
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024) · Nature · doi:10.1038/s41586-024-07487-w · Cited by 15,861 · cited 3× · see citing works
“Deep learning methods such as AlphaFold2 21 have revolutionized protein structure prediction, inspiring the development of all-atom cofolding methods like RoseTTAFold All-Atom 22 , AlphaFold3 23 , Chai-1 24 , Boltz-1 25 , Boltz-2 26 , Protenix 27 and NeuralPLexer3 28 .”
InfluentialFast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2022) · doi:10.1101/2022.02.07.479398 · Cited by 2,598 · see citing works
Fast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01773-0 · Cited by 2,596 · see citing works
High‐accuracy protein structure prediction in <scp>CASP14</scp>
Pereira, J., Simpkin, A. J., Hartmann, M. D., Rigden, D. J., Keegan, R. M., & Lupas, A. N. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26171 · Cited by 314 · see citing works
New prediction categories in <scp>CASP15</scp>
Kryshtafovych, A., Antczak, M., Szachniuk, M., Zok, T., Kretsch, R. C., Rangan, R., Pham, P., Das, R., Robin, X., Studer, G., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26515 · Cited by 54 · cited 1× · see citing works
“Recently, CASP15 had a pilot experiment in the evaluation of protein–ligand complex prediction 8 , inspired by previous efforts such as Teach Discover Treat 9 , Continuous Evaluation of Ligand Prediction Performance 10 , Drug Discovery Data Resource 11–14 and Community Structure–Activity Resource…”
Self-citationA fully automated benchmarking suite to compare macromolecular complexes
Studer, G., Robin, X., Bienert, S., Durairaj, J., Škrinjar, P., Tauriello, G., Waterhouse, A. M., & Schwede, T. (2026) · Nature Methods · doi:10.1038/s41592-025-02973-z · Cited by 10 · cited 4× · see citing works
“To evaluate the accuracy of predicted models, we use the compare-ligand-structures action of OpenStructure 34 (v.2.8.0), which returns the binding-site superposed symmetry-corrected ligand r.m.s.d. (hereafter referred as just r.m.s.d.), the LDDT-LP and the LDDT-PLI.”
InfluentialSelf-citation
Flagged first, then by citations
- Authors
- Škrinjar, P., Eberhardt, J., Studer, G., Tauriello, G., Schwede, T., & Durairaj, J.
- Affiliations
- SIB Swiss Institute of Bioinformatics (CH)
- Corresponding author
- Durairaj, Janani
- ORCID
- Venue
- Nature Structural & Molecular Biology
- Published
- 2026-05
- Volume
- 33
- Issue
- 5
- Pages
- 782-794
- ISSN
- 1545-9985 (ISSN-L)
- 1545-9993
- PMC
- PMC13661253
- OpenAlex
- W7160647050
- Cited by
- 22
- References
- 7
- Citations by year
- 2026: 22
- FWCI
- 45.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Nature Structural & Molecular Biology (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Universität Basel
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (223634)
- Swiss Institute of Bioinformatics
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Keywords
- protein-ligand complex prediction, benchmark dataset, generalization, memorization, de novo drug design, deep learning, protein-ligand interactions
- MeSH terms
- Deep Learning, Ligands, Models, Molecular, Protein Binding, Protein Conformation, Proteins, Drug Design, Protein Folding, Computational Biology, Drug Discovery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
