Cited by 344Protein Structure and DynamicsMachine Learning in BioinformaticsComputational Drug Discovery Methods

Critical assessment of protein intrinsic disorder prediction

Open access
Necci, Marco, Piovesan, Damiano, CAID Predictors, Hoque, Md Tamjidul, Walsh, Ian, Iqbal, Sumaiya, Vendruscolo, Michele, Sormanni, Pietro +92 more·2021·Nature Methods

Published version of a preprint

Abstract

Abstract Intrinsically disordered proteins, defying the traditional protein structure–function paradigm, are a challenge to study experimentally. Because a large part of our knowledge rests on computational predictions, it is crucial that their accuracy is high. The Critical Assessment of protein Intrinsic Disorder prediction (CAID) experiment was established as a community-based blind test to determine the state of the art in prediction of intrinsically disordered regions and the subset of residues involved in binding. A total of 43 methods were evaluated on a dataset of 646 proteins from DisProt. The best methods use deep learning techniques and notably outperform physicochemical methods. The top disorder predictor has F max = 0.483 on the full dataset and F max = 0.792 following filtering out of bona fide structured regions. Disordered binding regions remain hard to predict, with F max = 0.231. Interestingly, computing times among methods can vary by up to four orders of magnitude.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41592-021-01117-3

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • pLDDT scores have been shown to be a competitive disorder predictor compared to standards in disorder prediction.

    “pLDDT scores have been shown to be a competitive disorder predictor compared to standards in disorder prediction.5”
  • Conservative estimates indicate that roughly 30% of sequences, 30 residues or longer, drawn at random from the human proteome, are likely to be intrinsically disordered regions.

    “Conservative estimates indicate that roughly 30% of sequences, 30residues or longer, drawn at random from the human proteome, are likely to be IDRs. 3,5”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 58 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Necci, M., Piovesan, D., Predictors, C., Hoque, M. T., Walsh, I., Iqbal, S., Vendruscolo, M., Sormanni, P., Wang, C., Raimondi, D., Sharma, R., Zhou, Y., Litfin, T., Galzitskaya, O. V., Lobanov, M. Y., Vranken, W., Wallner, B., Mirabello, C., Malhis, N., Dosztányi, Z., Erdős, G., Mészáros, B., Gao, J., Wang, K., Hu, G., Wu, Z., Sharma, A., Hanson, J., Paliwal, K., Callebaut, I., Bitard-Feildel, T., Orlando, G., Peng, Z., Xu, J., Wang, S., Jones, D. T., Cozzetto, D., Meng, F., Yan, J., Gsponer, J., Cheng, J., Wu, T., Kurgan, L., Curators, D., Promponas, V. J., Tamana, S., Marino-Buslje, C., Martínez-Pérez, E., Chasapi, A., Ouzounis, C., Dunker, A. K., Kajava, A. V., Leclercq, J. Y., Aykac-Fas, B., Lambrughi, M., Maiani, E., Papaleo, E., Chemes, L. B., Álvarez, L., González-Foutel, N. S., Iglesias, V., Pujols, J., Ventura, S., Palopoli, N., Benítez, G. I., Parisi, G., Bassot, C., Elofsson, A., Govindarajan, S., Lamb, J., Salvatore, M., Hatos, A., Monzon, A. M., Bevilacqua, M., Mičetić, I., Minervini, G., Paladin, L., Quaglia, F., Leonardi, E., Davey, N., Horvath, T., Kovacs, O. P., Murvai, N., Pancsa, R., Schad, E., Szabo, B., Tantos, A., Macedo-Ribeiro, S., Manso, J. A., Pereira, P. J. B., Davidović, R., Veljkovic, N., Hajdu-Soltész, B., Pajkos, M., Szaniszló, T., Guharoy, M., Lazar, T., Macossay-Castillo, M., Tompa, P., & Tosatto, S. C. E.
Affiliations
  • University of Padua (IT)
  • LSU New Orleans (US)
  • Agency for Science, Technology and Research (SG)
  • Broad Institute (US)
  • University of Cambridge (GB)
  • Columbia University (US)
  • Fiji National University (FJ)
  • Griffith University (AU)
  • Russian Academy of Sciences (RU)
  • Linköping University (SE)
  • Eötvös Loránd University (HU)
  • Nankai University (CN)
  • Sorbonne Université (FR)
  • Tianjin University (CN)
  • Toyota Technological Institute at Chicago (US)
  • University College London (GB)
  • University of Alberta (CA)
  • University of British Columbia (CA)
  • University of Missouri (US)
  • Virginia Commonwealth University (US)
  • University of Cyprus (CY)
  • Fundación Instituto Leloir (AR)
  • Centre for Research and Technology Hellas (GR)
  • Université de Montpellier (FR)
  • Danish Cancer Society (DK)
  • Consejo Nacional de Investigaciones Científicas y Técnicas (AR)
  • Universitat Autònoma de Barcelona (ES)
  • National University of Quilmes (AR)
  • Stockholm University (SE)
  • University of Copenhagen (DK)
  • Institute of Cancer Research (GB)
  • Institute of Molecular Life Sciences (HU)
  • Universidade do Porto (PT)
  • Vrije Universiteit Brussel (BE)
Corresponding author
Tosatto, Silvio C. E.
Venue
Nature Methods
Published
2021-04-19
Volume
18
Issue
5
Pages
472-481
ISSN
  • 1548-7091 (ISSN-L)
  • 1548-7105
OpenAlex
W3156051371
Cited by
343
References
58
Citations by year
2026: 35 · 2025: 64 · 2024: 68 · 2023: 72 · 2022: 69 · 2021: 32 · 2020: 3
FWCI
18.99
Type
journal article
Language
en
Open access
Locations
  • Nature Methods (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona) (submittedVersion, cc-by)
  • Griffith Research Online (submittedVersion, cc-by)
  • CONICET Digital (CONICET) (submittedVersion, cc-by)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • ELTE Digital Institutional Repository (EDIT) (Eötvös Loránd University) (submittedVersion, other-oa)
  • UCL Discovery (University College London) (submittedVersion)
  • IUScholarWorks (Indiana University) (submittedVersion, cc-by)
  • VinaR (Institute of Nuclear Sciences "Vinča") (submittedVersion, cc-by)
  • VUBIR (Vrije Universiteit Brussel) (submittedVersion)
  • Padua Research Archive (University of Padova) (submittedVersion, cc-by)
  • Apollo
  • Apollo (cc-by)
Funders
  • EC | Horizon 2020 Framework Programme (778247)
  • Ministero dell’Istruzione, dell’Università e della Ricerca (2017483NH8)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
Keywords
intrinsically disordered proteins, CAID, blind test, deep learning, physicochemical methods, DisProt, prediction accuracy, benchmarking
MeSH terms
Amino Acid Sequence, Protein Binding, Protein Conformation, Software, Protein Folding, Computational Biology, Databases, Protein, Intrinsically Disordered Proteins

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review