CoChat
Sign Up
Cited by 1Protein Structure and DynamicsChemical Synthesis and AnalysisRNA and protein synthesis mechanisms

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Scaffold-Lab: Critical evaluation and ranking of protein backbone generation methods in a unified framework

Open access
Zheng, Zhuoqi, Zhang, Bo, Zhong, Bozitao, Yu, Jinyu, Liu, Kexin, Li, Zhengxin, Zhu, Junjie, Wei, Ting, Chen, Hai-Feng·2026·PLOS Computational Biology
DOI 10.1371/journal.pcbi.1014290

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advances in de novo protein design have significantly progressed, particularly in protein backbone generation, which remains a challenging yet valuable task. However, there has been a lack in standardized evaluation framework to accurately assess diverse methods, nor is there a comprehensive analysis of their practical applications. In this study, we proposed Scaffold-Lab, a unified framework for systematic evaluation of protein backbone generation methods, encompassing a wide range of metrics including designability, novelty, diversity, efficiency and structural properties. We thoroughly evaluated seven representative methods, providing a detailed analysis of their performance and utilities. Our findings highlight that generating long proteins for unconditional generation and accurately reconstructing motifs are key bottlenecks for most methods. These results underscore both the strengths and limitations of existing protein backbone generation methods, offering new insights for further development and improvement in this field.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1014290

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 80

  1. CB Anfinsen, 1973, Principles that govern the folding of protein chains, Science, 181, 223, 10.1126/science.181.4096.223 · DOI · Paper page
  2. K Dill, 2017, Protein actions: Principles and modeling, 10.1201/9781315212210 · DOI
  3. P-S Huang, 2016, The coming of age of de novo protein design, Nature, 537, 320, 10.1038/nature19946 · DOI · Paper page
  4. G Casadevall, 2023, AlphaFold2 and Deep Learning for Elucidating Enzyme Conformational Flexibility and Its Application for Design, JACS Au, 3, 1554, 10.1021/jacsau.3c00188 · DOI
  5. H Khakzad, 2023, A new age in protein design empowered by deep learning, Cell Syst, 14, 925, 10.1016/j.cels.2023.10.006 · DOI
  6. B Kuhlman, 2003, Design of a novel globular protein fold with atomic-level accuracy, Science, 302, 1364, 10.1126/science.1089427 · DOI
  7. J Mu, 2023, De novo protein sequence design based on deep learning and validation on CalB hydrolase, bioRxiv
  8. N Ferruz, 2022, ProtGPT2 is a deep unsupervised language model for protein design, Nat Commun, 13, 4348, 10.1038/s41467-022-32007-7 · DOI · Paper page
  9. N Anand, 2022, Protein sequence design with a learned potential, Nat Commun, 13, 746, 10.1038/s41467-022-28313-9 · DOI · Paper page
  10. Z Zheng, 2023, Structure-informed language models are protein designers, bioRxiv
  11. J Ingraham, 2019, Generative models for graph-based protein design, Advances in neural information processing systems, 32
  12. B Jing, 2020, Learning from protein structure with geometric vector perceptrons. arXiv preprint
  13. M Mirdita, 2022, ColabFold: making protein folding accessible to all, Nat Methods, 19, 679, 10.1038/s41592-022-01488-1 · DOI · Paper page
  14. AW Senior, 2020, Improved protein structure prediction using potentials from deep learning, Nature, 577, 706, 10.1038/s41586-019-1923-7 · DOI · Paper page
  15. W Wang, 2022, Single-sequence protein structure prediction using supervised transformer protein language models, Nat Comput Sci, 2, 804, 10.1038/s43588-022-00373-3 · DOI · Paper page
  16. B Zhong, 2021, ParaFold: Paralleling AlphaFold for Large-Scale Predictions. arXiv preprint
  17. B Huang, 2022, A backbone-centred energy function of neural networks for protein design, Nature, 602, 523, 10.1038/s41586-021-04383-5 · DOI
  18. J Ho, 2020, Denoising diffusion probabilistic models, Advances in Neural Information Processing Systems, 33, 6840
  19. Z Guo, 2024, Diffusion models in bioinformatics and computational biology, Nat Rev Bioeng, 2, 136, 10.1038/s44222-023-00114-9 · DOI
  20. N Ferruz, 2022, From sequence to function through structure: Deep learning for protein design, Comput Struct Biotechnol J, 21, 238, 10.1016/j.csbj.2022.11.014 · DOI
  21. N Anand, 2022, Protein structure and sequence generation with equivariant denoising diffusion probabilistic models. arXiv preprint
  22. AE Chu, 2023, An all-atom protein generative model, bioRxiv
  23. B 32Ni, 2023, ForceGen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a protein language diffusion model. arXiv preprint
  24. BL Trippe, 2022, Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. arXiv preprint
  25. LV Castorina, 2023, PDBench: evaluating computational methods for protein-sequence design, Bioinformatics, 39, 10.1093/bioinformatics/btad027 · DOI
  26. Z Gao, 2023, in Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  27. J Yu, 2024, Multi-indicator comparative evaluation for deep learning-based protein sequence design methods, Bioinformatics
  28. C Wang, 2023, PDB-Struct: A Comprehensive Benchmark for Structure-based Protein Design. arXiv preprint
  29. P Notin, 2023, ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction, bioRxiv
  30. M Buttenschoen, 2023, PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences, Chem Sci, 15, 3130, 10.1039/D3SC04185A · DOI
  31. L Jiang, 2008, De Novo Computational Design of Retro-Aldol Enzymes, Science, 319, 1387, 10.1126/science.1152692 · DOI
  32. SV Torres, 2023, De novo design of high-affinity binders of bioactive helical peptides, Nature
  33. E Procko, 2014, A computationally designed inhibitor of an Epstein-Barr viral Bcl-2 protein induces apoptosis in infected cells, Cell, 157, 1644, 10.1016/j.cell.2014.04.034 · DOI · Paper page
  34. J Wang, 2022, Scaffolding protein functional sites using deep learning, Science, 377, 387, 10.1126/science.abn2100 · DOI · Paper page
  35. S Alamdari, 2023, Protein generation with evolutionary diffusion: sequence is all you need, bioRxiv
  36. B Zhang, 2023, Protein language model supervised precise and efficient protein backbone design method, bioRxiv
  37. Y Lin, 2023, Generating novel, designable, and diverse protein structures by equivariantly diffusing oriented residue clouds. arXiv preprint
  38. J Yim, 2023, Fast protein backbone generation with SE (3) flow matching. arXiv preprint
  39. J Yim, 2023, SE (3) diffusion model with application to protein backbone generation. arXiv preprint
  40. KH Sumida, 2024, Improving Protein Expression, Stability, and Function with ProteinMPNN, Journal of the American Chemical Society
  41. C Hsu, 2022, Learning inverse folding from millions of predicted structures, bioRxiv
  42. AM Hermosilla, 2024, Validation of de novo designed water-soluble and transmembrane β-barrels by in silico folding and melting, Protein Sci, 33, 10.1002/pro.5033 · DOI
  43. C Frank, 2024, Scalable protein design using optimization in a relaxed sequence space, Science, 386, 439, 10.1126/science.adq1741 · DOI
  44. R Wu, 2022, High-resolution de novo structure prediction from primary sequence, bioRxiv
  45. Y Zhang, 2004, Scoring function for automated assessment of protein structure template quality, Proteins, 57, 702, 10.1002/prot.20264 · DOI · Paper page
  46. P Eastman, 2013, OpenMM 4: A Reusable, Extensible, Hardware Independent Library for High Performance Molecular Simulation, J Chem Theory Comput, 9, 461, 10.1021/ct300857j · DOI
  47. H Li, 1996, Emergence of preferred structures in a simple model of protein folding, Science, 273, 666, 10.1126/science.273.5275.666 · DOI
  48. F Pan, 2021, Estimating the designability of protein structures, bioRxiv
  49. R Mélin, 1999, Designability, thermodynamic stability, and dynamics in protein folding: A lattice model study, The Journal of Chemical Physics, 110, 1252, 10.1063/1.478168 · DOI
  50. H Li, 1998, Are protein folds atypical?, Proc Natl Acad Sci U S A, 95, 4987, 10.1073/pnas.95.9.4987 · DOI
  51. P Virtanen, 2020, SciPy 1.0: fundamental algorithms for scientific computing in Python, Nat Methods, 17, 261, 10.1038/s41592-019-0686-2 · DOI · Paper page
  52. SK Burley, 2023, RCSB Protein Data Bank (RCSB.org): delivery of experimentally-determined PDB structures alongside one million computed structure models of proteins from artificial intelligence/machine learning, Nucleic Acids Res, 51, 10.1093/nar/gkac1077 · DOI
  53. H Berman, 2003, Announcing the worldwide Protein Data Bank, Nature Structural & Molecular Biology, 10, 980, 10.1038/nsb1203-980 · DOI · Paper page
  54. M van Kempen, 2024, Fast and accurate protein structure search with Foldseek, Nat Biotechnol, 42, 243, 10.1038/s41587-023-01773-0 · DOI · Paper page
  55. J Mu, 2021, Recent Force Field Strategies for Intrinsically Disordered Proteins, J Chem Inf Model, 61, 1037, 10.1021/acs.jcim.0c01175 · DOI
  56. CJ Williams, 2018, MolProbity: More and better reference data for improved all-atom structure validation, Protein Sci, 27, 293, 10.1002/pro.3330 · DOI · Paper page
  57. GH Tzeng, 2011, Multiple attribute decision making: methods and applications, 10.1201/b11032 · DOI
  58. TL Saaty, 1988, What is the analytic hierarchy process?
  59. D Diakoulaki, 1995, Determining objective weights in multiple criteria problems: The critic method, Computers & Operations Research, 22, 763, 10.1016/0305-0548(94)00059-H · DOI
  60. S Xiang, 2013, Phosphorylation drives a dynamic switch in serine/arginine-rich proteins, Structure, 21, 2162, 10.1016/j.str.2013.09.014 · DOI
  61. YGJ Sterckx, 2014, Small-Angle X-Ray Scattering- and Nuclear Magnetic Resonance-Derived Conformational Ensemble of the Highly Flexible Antitoxin PaaA2, Structure, 22, 854, 10.1016/j.str.2014.03.012 · DOI
  62. M Corbella, 2023, Loop dynamics and the evolution of enzyme activity, Nat Rev Chem, 7, 536, 10.1038/s41570-023-00495-w · DOI
  63. J Zhu, 2025, Accurate Generation of Conformational Ensembles for Intrinsically Disordered Proteins with IDPFold, Adv Sci (Weinh), 12
  64. R Obexer, 2017, Emergence of a catalytic tetrad during evolution of a highly active artificial aldolase, Nat Chem, 9, 50, 10.1038/nchem.2596 · DOI
  65. M Braun, 2026, Computational enzyme design by catalytic motif scaffolding, Nature, 649, 237, 10.1038/s41586-025-09747-9 · DOI · Paper page
  66. L Chang, 2024, Revolutionizing Peptide-Based Drug Discovery: Advances in the Post-AlphaFold Era, Wiley Interdiscip Rev Comput Mol Sci, 14, 10.1002/wcms.1693 · DOI · Paper page
  67. NR Bennett, 2023, Improving de novo protein binder design with deep learning, Nat Commun, 14, 2625, 10.1038/s41467-023-38328-5 · DOI · Paper page
  68. P Gainza, 2023, De novo design of protein interactions with learned surface fingerprints, Nature, 617, 176, 10.1038/s41586-023-05993-x · DOI
  69. O Khersonsky, 2022, What Have We Learned from Design of Function in Large Proteins?, Biodes Res, 2022, 9787581, 10.34133/2022/9787581 · DOI
  70. BIM Wicky, 2022, Hallucinating symmetric protein assemblies, Science, 378, 56, 10.1126/science.add1964 · DOI · Paper page
  71. DSG 89and, 2000, Structural symmetry and protein function, Annual Review of Biophysics and Biomolecular Structure, 29, 105, 10.1146/annurev.biophys.29.1.105 · DOI · Paper page
  72. L An, 2023, De novo design of diverse small molecule binders and sensors using Shape Complementary Pseudocycles, bioRxiv
  73. SA Rettie, 2023, Cyclic peptide structure prediction and design using AlphaFold, bioRxiv
  74. AE Chu, 2024, Sparks of function by de novo protein design, Nat Biotechnol, 42, 203, 10.1038/s41587-024-02133-2 · DOI
  75. C Frank, 2023, Efficient and scalable de novo protein design using a relaxed sequence space, bioRxiv
  76. JR Jeliazkov, 2023, ESMFold hallucinates native-like protein sequences, bioRxiv
  77. X Ji, 2023, Personal Precise Force Field for Intrinsically Disordered and Ordered Proteins Based on Deep Learning, J Chem Inf Model, 63, 362, 10.1021/acs.jcim.2c01501 · DOI
  78. X Cui, 2022, Polarizable Force Field of Intrinsically Disordered Proteins with CMAP and Reweighting Optimization, J Chem Inf Model, 62, 4970, 10.1021/acs.jcim.2c00835 · DOI
  79. G Song, 2023, Phosphorylation Modification Force Field FB18CMAP Improving Conformation Sampling of Phosphoproteins, Journal of Chemical Information and Modeling, 63, 1602, 10.1021/acs.jcim.3c00112 · DOI
  80. J Zhu, 2023, Phanto-IDP: compact model for precise intrinsically disordered protein backbone generation and enhanced sampling, Brief Bioinform, 25

Metadata

Authors
Zheng, Z., Zhang, B., Zhong, B., Yu, J., Liu, K., Li, Z., Zhu, J., Wei, T., & Chen, H.-F.
Affiliations
Shanghai Jiao Tong University (CN)
Corresponding author
Wei, Ting
ORCID
Venue
PLOS Computational Biology
Publisher
Public Library of Science (PLoS)
Published
2026-05-22
Volume
22
Issue
5
Pages
e1014290
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
DOI
10.1371/journal.pcbi.1014290
OpenAlex
W4391757211
Cited by
1
References
15
Citations by year
2026: 1
FWCI
2.05
Type
journal article
Language
en
Update notices
new_version (2026-06-01)
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • SSRN Electronic Journal (acceptedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Key Research and Development Program of China (2025YFA0921000 and 2023YFF1205102)
  • National Natural Science Foundation of China (32571435)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Physical Sciences › Chemistry › Organic Chemistry › Chemical Synthesis and Analysis
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
Keywords
de novo protein design, benchmarking, unconditional generation, conditional generation, designability, novelty, diversity, RFdiffusion, protein engineering
MeSH terms
Algorithms, Models, Molecular, Protein Conformation, Proteins, Software, Protein Engineering, Computational Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review