Assessing the accuracy of contact predictions in CASP13
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The accuracy of sequence‐based tertiary contact predictions was assessed in a blind prediction experiment at the CASP13 meeting. After 4 years of significant improvements in prediction accuracy, another dramatic advance has taken place since CASP12 was held 2 years ago. The precision of predicting the top L/5 contacts in the free modeling category, where L is the corresponding length of the protein in residues, has exceeded 70%. As a comparison, the best‐performing group at CASP12 with a 47% precision would have finished below the top 1/3 of the CASP13 groups. Extensively trained deep neural network approaches dominate the top performing algorithms, which appear to efficiently integrate information on coevolving residues and interacting fragments or possibly utilize memories of sequence similarities and sometimes can deliver accurate results even in the absence of virtually any target specific evolutionary information. If the current performance is evaluated by F‐score on L contacts, it stands around 24% right now, which, despite the tremendous impact and advance in improving its utility for structure modeling, also suggests that there is much room left for further improvement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment
Remmert, M., Biegert, A., Hauser, A., & Söding, J. (2012) · Nature Methods · doi:10.1038/nmeth.1818 · Cited by 2,288
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434
Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
Wang, S., Sun, S., Li, Z., Zhang, R., & Xu, J. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005324 · Cited by 931 · cited 1×
“After 2010, when first papers on correlated mutations were published, a number of refinements took place, such as combining a set of nonoverlapping contact predictions in a consensus approach and especially the application of various supervised deep neural network based approaches,(42) further improving the accuracy o…”
BackgroundShindyalov, I. N., Kolchanov, N. A., & Sander, C. (1994) · "Protein Engineering, Design and Selection" · doi:10.1093/protein/7.3.349 · Cited by 297
Origins of coevolution between residues distant in protein 3D structures
Anishchenko, I., Ovchinnikov, S., Kamisetty, H., & Baker, D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1702664114 · Cited by 185 · cited 1×
“28-33 A significant advance took place when various approaches focused on addressing the transitivity problem in residue contact signals, such as the direct coupling analysis, 34-38 sparse inverse covariance methods, 37,39 and network decomposition approaches.”
BackgroundAssessment of contact predictions in CASP12: Co‐evolution and deep learning coming of age
Schaarschmidt, J., Monastyrskyy, B., Kryshtafovych, A., & Bonvin, A. M. J. J. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25407 · Cited by 177 · cited 2×
“As early as the second CASP meeting in 1996,(2,3) contact predictions were made and assessed.(4-13) In general, protein structures are very well packed,(14) with densities comparable”
BackgroundMethodSelf-citationLi, Y., Hu, J., Zhang, C., Yu, D.-J., & Zhang, Y. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz291 · Cited by 158
Enhancing Evolutionary Couplings with Deep Convolutional Neural Networks
Liu, Y., Palmedo, P., Ye, Q., Berger, B., & Peng, J. (2018) · Cell Systems · doi:10.1016/j.cels.2017.11.014 · Cited by 114
Cited by
Zheng, W., Zhang, C., Li, Y., Pearce, R., Bell, E. W., & Zhang, Y. (2021) · Cell Reports Methods · doi:10.1016/j.crmeth.2021.100014 · Cited by 569
Rao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., & Rives, A. (2021) · doi:10.1101/2021.02.12.430858 · Cited by 350
Deep learning methods in protein structure prediction
Torrisi, M., Pollastri, G., & Le, Q. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2019.12.011 · Cited by 284
Improved protein structure prediction by deep learning irrespective of co-evolution information
(2021) · doi:10.1038/s42256-021-00348-5 · Cited by 256
Convolutions are competitive with transformers for protein sequence pretraining
Yang, K. K., Fusi, N., & Lu, A. X. (2024) · Cell Systems · doi:10.1016/j.cels.2024.01.008 · Cited by 137
Metadata
- Authors
- Shrestha, R., Fajardo, E., Gil, N., Fidelis, K., Kryshtafovych, A., Monastyrskyy, B., & Fiser, A.
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2019-10-24
- Volume
- 87
- Issue
- 12
- Pages
- 1058-1068
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- OpenAlex
- W2977360892
- Cited by
- 62
- References
- 9
- FWCI
- 4.08
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.