CoChat

PDBench: evaluating computational methods for protein-sequence design

Open access
Castorina, Leonardo V, Petrenas, Rokas, Subr, Kartic, Wood, Christopher W · 2023 · Bioinformatics
Cited by 21

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

Summary: Ever increasing amounts of protein structure data, combined with advances in machine learning, have led to the rapid proliferation of methods available for protein-sequence design. In order to utilize a design method effectively, it is important to understand the nuances of its performance and how it varies by design target. Here, we present PDBench, a set of proteins and a number of standard tests for assessing the performance of sequence-design methods. PDBench aims to maximize the structural diversity of the benchmark, compared with previous benchmarking sets, in order to provide useful biological insight into the behaviour of sequence-design methods, which is essential for evaluating their performance and practical utility. We believe that these tools are useful for guiding the development of novel sequence design algorithms and will enable users to choose a method that best suits their design target. Availability and implementation: https://github.com/wells-wood-research/PDBench Supplementary information: Supplementary data are available at Bioinformatics online.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btad027

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 19

  1. Das, 2008, Macromolecular modeling with rosetta, Annu. Rev. Biochem, 77, 363, 10.1146/annurev.biochem.77.062906.171838 · DOI · Paper page
  2. Goldenzweig, 2016, Automated structure- and sequence-based design of proteins for high bacterial expression and stability, Mol. Cell, 63, 337, 10.1016/j.molcel.2016.06.012 · DOI · Paper page
  3. Henikoff, 1992, Amino acid substitution matrices from protein blocks, Proc. Natl. Acad. Sci. USA, 89, 10915, 10.1073/pnas.89.22.10915 · DOI · Paper page
  4. Huang, 2019, Convolutional networks with dense connectivity, IEEE Trans. Pattern Anal. Mach. Intell, 8704
  5. Huang, 2019, EvoEF2: Accurate and fast energy function for computational protein design, Bioinformatics, 36, 1135, 10.1093/bioinformatics/btz740 · DOI · Paper page
  6. Joosten, 2011, A series of PDB related databases for everyday needs, Nucleic Acids Res, 39, D411, 10.1093/nar/gkq1105 · DOI · Paper page
  7. Knudsen, 2010, The CATH database, Hum. Genomics, 4, 207, 10.1186/1479-7364-4-3-207 · DOI · Paper page
  8. Ludwiczak, 2018, Combining Rosetta with molecular dynamics (MD): A benchmark of the MD-based ensemble protein design, J. Struct. Biol, 203, 54, 10.1016/j.jsb.2018.02.004 · DOI
  9. O'Connell, 2018, Spin2: Predicting sequence profiles from protein structures using deep neural networks, Proteins, 86, 629, 10.1002/prot.25489 · DOI · Paper page
  10. Ollikainen, 2013, Computational protein design quantifies structural constraints on amino acid covariation, PLoS Comput. Biol, 9, e1003313, 10.1371/journal.pcbi.1003313 · DOI
  11. Qi, 2020, DenseCPD: Improving the accuracy of neural-network-based computational protein sequence design with DenseNet, J. Chem. Inf. Model, 60, 1245, 10.1021/acs.jcim.0c00043 · DOI · Paper page
  12. Stam, 2021, DE-STRESS: A user-friendly web application for the evaluation of protein designs, Protein Eng. Des. Sel, 34, 10.1093/protein/gzab029 · DOI
  13. Strokach, 2020, Fast and flexible protein design using deep graph neural networks, Cell Syst, 11, 402, 10.1016/j.cels.2020.08.016 · DOI · Paper page
  14. Wood, 2017, ISAMBARD: An open-source computational environment for biomolecular analysis, modelling and design, Bioinformatics, 33, 3043, 10.1093/bioinformatics/btx352 · DOI · Paper page
  15. Woolfson, 2021, A brief history of de novo protein design: Minimal, rational, and computational, J. Mol. Biol, 433, 167160, 10.1016/j.jmb.2021.167160 · DOI · Paper page
  16. Woolfson, 2015, De novo protein design: How do we expand into the universe of possible protein structures?, Curr. Opin. Struct. Biol, 33, 16, 10.1016/j.sbi.2015.05.009 · DOI · Paper page
  17. Wu, 2022, High-resolution de novo structure prediction from primary sequence, bioRxiv
  18. Yue, 1992, Inverse protein folding problem: Designing polymer sequences, Proc. Natl. Acad. Sci. USA, 89, 4163, 10.1073/pnas.89.9.4163 · DOI · Paper page
  19. Zhang, 2020, ProDCoNN: Protein design using a convolutional neural network, Proteins, 88, 819, 10.1002/prot.25868 · DOI · Paper page

Cited by

See all 21

Metadata

Authors
Castorina, L. V., Petrenas, R., Subr, K., & Wood, C. W.
Venue
Bioinformatics
Published
2023-01-01
Volume
39
Issue
1
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
OpenAlex
W3199169486
Cited by
21
References
15
FWCI
1.34
Type
journal article
Language
en
Open access
gold
Funders
  • Engineering and Physical Sciences Research Council (EP/S003002/1)
  • UK Research and Innovation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.