CoChat
Sign Up
Cited by 722Protein Structure and DynamicsEnzyme Structure and FunctionMachine Learning in Bioinformatics

A structural biology community assessment of AlphaFold2 applications

Open access
Akdel, Mehmet, Pires, Douglas E. V., Pardo, Eduard Porta, Jänes, Jürgen, Zalevsky, Arthur O., Mészáros, Bálint, Bryant, Patrick, Good, Lydia L. +26 more·2022·Nature Structural & Molecular Biology
DOI 10.1038/s41594-022-00849-wPMC PMC9663297
Open this paper in CoChat — free accountOpen PDF

Abstract

Most proteins fold into 3D structures that determine how they function and orchestrate the biological processes of the cell. Recent developments in computational methods for protein structure predictions have reached the accuracy of experimentally determined models. Although this has been independently verified, the implementation of these methods across structural-biology applications remains to be tested. Here, we evaluate the use of AlphaFold2 (AF2) predictions in the study of characteristic structural elements; the impact of missense variants; function and ligand binding site predictions; modeling of interactions; and modeling of experimental structural data. For 11 proteomes, an average of 25% additional residues can be confidently modeled when compared with homology modeling, identifying structural features rarely seen in the Protein Data Bank. AF2-based predictions of protein disorder and complexes surpass dedicated tools, and AF2 models can be used across diverse applications equally well compared with experimentally determined structures, when the confidence metrics are critically considered. In summary, we find that these advances are likely to have a transformative impact in structural biology and broader life-science research.

AI snapshot

Written from the paper’s full text.

TL;DR

This community assessment evaluates AlphaFold2 (AF2) across structural-biology applications, including proteome coverage, disorder, variant effects, pockets, complexes, and experimental model building. It finds AF2 models are broadly comparable to experimental structures when confidence metrics are critically considered, while low-confidence regions and some high-confidence details still require caution.

Key findings

  • For 11 proteomes shared with the SWISS-MODEL Repository, AF2 added an average of 44% residue coverage; among residues absent from SMR, 49.4% had pLDDT > 70, and on average around 25% of proteome residues were covered by novel, confident AF2 predictions. Across the 11 species, 18,429 contiguous “domain-like” regions (100–500 residues, pLDDT > 70) had no SMR model.
  • AF2-derived disorder metrics (pLDDT and window averages of pLDDT or SASA) outperformed IUPred2. For oligomeric state prediction, 71/87 (82%) top-scoring models matched the correct state; for heterodimeric docking, 31% of AF2 models were correct versus 7% for GRAMM.
  • In variant-effect tests, AF2-based predictions of mutation impact consistently matched or were better than those from experimental models for 117,135 DMS mutations in 33 proteins; stability predictions across 2,648 mutations in 121 proteins were comparable to experimental structures, while homology models degraded below 40% sequence identity.
  • High-quality AF2 pockets (mean pLDDT > 90) performed similarly to pockets from experimental structures, whereas low-confidence pockets generally did not overlap known sites. AF2 also revealed a novel beta-solenoid for pentapeptide repeats, proposed as the shortest repeat length forming such a structure.

Methods

  • The paper is a multi-application benchmark/evaluation study using AF2 predictions from public databases and locally generated models, compared against experimental structures, homology models, SMR, trRosetta/Pfam models, and PDB entries.
  • Datasets included 11 model proteomes, 365,198 AF2 proteins, 104,323 PDB proteins, 117,135 DMS mutations in 33 proteins, 2,648 stability mutations in 121 proteins, 225 proteins with known binding sites, 87 oligomeric proteins, 219 heterodimeric complexes, and 14 IDR-containing complexes.
  • Analyses used structure-based predictors (FoldX, Rosetta, DynaMut2), disorder benchmarking against IUPred2, Geometricus shape-mers with NMF/t-SNE, AutoSite pocket detection, DockQ for docking, and ISOLDE/phenix for cryo-EM and crystallographic model building.

Limitations

  • The paper states that AF2 models can contain uncertainly placed segments, leading to incorrect structural similarity, pocket, variant-effect, or model-building estimates; confidence metrics must be incorporated into workflows. It also notes that even high-confidence regions can contain subtle errors, citing Trp A111 (pLDDT = 86.1) modeled backwards.
  • The authors state that low-confidence predictions are enriched for disorder, but caution this may be partly a by-product of AF2 being trained primarily on X-ray data; they also note IUPred2 is not the best-performing dedicated disorder predictor.
  • The paper states open questions remain about generalization to truly unseen folds, prediction of mutated protein structures, conformational diversity/dynamics, structure without an MSA, and complexes with DNA, RNA, or metabolites. Evident from the text: several benchmarks rely on specific released databases and curated sets, and some oligomer examples may overlap AF2 training even though the authors argue this still tests oligomeric-state prediction.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41594-022-00849-w

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 35 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    For the 11 model species, an average of around 25% of proteome residues are covered by AF2 with novel (not present in SMR) and confident (pLDDT > 70) predictions.

    “For 11 proteomes, an average of 25% additional residues can be confidently modeled when compared with homology modeling, identifying structural features rarely seen in the Protein Data Bank.”
    • Also stated in: § Added structural coverage by AlphaFold2 predictions of model proteomes · plain
      “In summary, an average of around 25% of the residues of the proteomes of the 11 model species are covered by AF2 with novel (not present in SRM) and confident (pLDDT > 70) predictions.”
    Quote in the paperSubstantial claim
  • Findingnone given

    AF2-based predictions of protein disorder surpass dedicated tools.

    “AF2-based predictions of protein disorder and complexes surpass dedicated tools, and AF2 models can be used across diverse applications equally well compared with experimentally determined structures, when the confidence metrics are critically considered.”
    Quote in the paperSubstantial claim
  • Findingown data

    AF2 predictions identify structural features rarely seen in the Protein Data Bank.

    “For 11 proteomes, an average of 25% additional residues can be confidently modeled when compared with homology modeling, identifying structural features rarely seen in the Protein Data Bank.”
    Quote in the paperSubstantial claim
  • Findingown data

    For 11 model species included in both SMR and AF2 databases, AF2 provides an average additional coverage of 44% of residues.

    “We compared the structures of 11 model species that were included in both the SMR and AF2 databases and that had an average additional coverage of 44% of residues by AF2 (Fig. 1a, residues).”
    Quote in the paperSubstantial claim
  • Findingown data

    With a stringent cut-off (pLDDT > 90), AF2 predicts, on average, 25% of residues with very high confidence.

    “With a more stringent cut-off (pLDDT > 90), AF2 predicts, on average, 25% of residues with very high confidence.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 34
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 14

  • May combine two claims.

    Findingcitation

    The disease-associated missense variant p.A280V in DEGS1 has been shown to lead to loss of protein stability and has a predicted ΔΔG of 3.7 kcal/mol.

    “p.A280V has been shown to lead to loss of protein stability34 and has a predicted Gibbs free energy change (ΔΔG) of 3.7 kcal/mol.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] Karsai, G. et al. DEGS1-associated aberrant sphingolipid metabolism impairs nervous system function in humans. J. Clin. Invest. 129, 1229–1239 (2019). source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    Topic 38 consists of short fragments of PDB structures, with an average length of 63 residues; there are no AF2 proteins because AlphaFold models the entire structure instead of returning fragments.

    “Topic 38 consists of short fragments of PDB structures, with an average length of 63 residues—there are no AF2 proteins, because AlphaFold models the entire structure instead of returning fragments.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Pockets predicted from AF2 structures also have a wider size range than ground-truth binding sites, including a small number of particularly large pockets.

    “This is also true for pockets predicted from AF2 structures, including a small number of particularly large pockets (Fig. 4a).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The AF2 database represents around twice the number of experimental structures and six times the number of unique proteins in the PDB.

    “This represents around twice the number of experimental structures and six times the number of unique proteins in the Protein Data Bank (PDB).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Out of 250 total groups, 5 examples were almost exclusively (>90%) composed of AF2-derived structures, and 1 example had >80% AF2 structures with a particularly interesting novel predicted structural element.

    “Out of 250 total groups, we selected 5 examples that were almost exclusively (>90%) composed of structures derived from AF2, as well as 1 example with >80% AF2 structures with a particularly interesting novel predicted structural element.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingnone given

    The 20 most common superfamilies, predicted from sequence, tend to be placed together in the shape-mer space.

    “Additionally, the 20 most common superfamilies, predicted from sequence, tend to be placed together.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    pLDDT and window averages of pLDDT or SASA outperform IUPred2 in disorder prediction, indicating that AF2's low-confidence predictions are enriched for IDRs.

    “pLDDT and window averages of pLDDT or SASA outperformed IUPred2, indicating that AF2’s low-confidence predictions are enriched for IDRs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Low-quality AF2 pockets (mean pLDDT ≤ 90) are larger on average and include particularly large pockets.

    “Low-quality pockets are larger on average, and include particularly large pockets (Fig. 4a, bottom).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    As many antibodies are under intense study, there are many more PDB structures than the actual number of such proteins in the respective UniProt proteomes.

    “As many of these antibodies are under intense study, there are many more PDB structures (based on multiple individuals and antibody-drug research) than the actual number of such proteins in the respective UniProt proteomes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    AF2 performance on the Dockground 4.3 heterodimeric benchmark is far superior to traditional docking methods, with 31% correctly predicted protein complex models compared with 7% using GRAMM.

    “In Figure 5e, it can be seen that the performance is far superior to traditional docking methods, with 31% of correctly predicted protein complex models, compared with 7% using GRAMM, a standard shape-complementarity docking method44.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [44] Tovchigrechko, A., Wells, C. A. & Vakser, I. A. Docking of protein models. Protein Sci. 11, 1888–1896 (2002). source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    For the RelA-CBP interaction, AF2 correctly finds the binding groove but fits the IDR in a reverse orientation.

    “For the RelA–CBP interaction, AF2 correctly finds the binding groove, but fits the IDR in a reverse orientation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The final rebuilt model increased the number of modeled, identified residues from 600 to 818, slightly improved overall geometry, and reduced the Rfree from 0.317 to 0.295.

    “The total time spent was approximately one working day; the final model (Fig. 6f–h) increased the number of modeled, identified residues from 600 to 818, slightly improved overall geometry and reduced the Rfree from 0.317 to 0.295.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    For some cases of failed tetramer predictions, higher confidence of tetramer predictions could be obtained by increasing the number of recycles.

    “We noted that, for some cases of failed tetramer predictions, we could obtain higher confidence of the tetramer predictions by increasing the number of recycles.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Rigid-body alignment of AF2 models to corresponding experimental chains of eIF2B showed overall excellent agreement, with the largest deviations corresponding to correctly folded domains with flexible connections to neighbors.

    “Rigid-body alignment of AF2 models to their corresponding experimental chains (Fig. 6a) showed overall excellent agreement, with the largest deviations corresponding to correctly folded domains with flexible connections to their neighbors.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    DEGS1 (UniProt O15121) has high confidence (average pLDDT = 96.31) and no previous structural data.

    “we focused on the human sphingolipid delta(4)-desaturase (EC 1.14.19.17, DEGS1, UniProt Accession O15121, pocket score rank 57 of 912), which has a high confidence level (average pLDDT = 96.31) and for which there are no previous structural data.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    The prediction score of the highest ranked pocket enriched the set of 912 human proteins with no experimental or homology models for proteins with previous annotations for enzymatic activity.

    “We found that the prediction score of the highest ranked pocket enriched the set for proteins with previous annotations for enzymatic activity (Fig. 4c and Supplementary Table 3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    AF2 returns full protein predictions, but these can often contain protein segments that are placed with uncertainty.

    “Although AF2 returns full protein predictions, these can often contain protein segments that are placed with uncertainty.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    A sequence search of the 323-residue DEGS1 against all PDB entries shows the best sequence match is 23.5%, with PDB entry 1VHB, indicating lack of structural models from homology.

    “A sequence search of the 323-residue protein against all existing entries in the PDB shows that the best sequence match is 23.5%, with PDB entry 1VHB (Bacterial dimeric hemoglobin, 9115439), indicating the lack of any structural models from homology.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Although the failure rate is high for tetramer state predictions, the predicted structure for the corresponding state was actually correct for 5/6 failures.

    “Notably, although the failure rate is high for tetramer state predictions, the predicted structure for the corresponding state was actually correct for 5/6 failures.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Across the oligomerization set, higher scores were given to models corresponding to the correct oligomerization state, and 71 out of 87 (82%) predicted top-scoring models corresponded to the correct state.

    “Across the set of predictions, higher scores were given to models corresponding to the correct oligomerization state, and 71 out of 87 (82%) predicted top-scoring models corresponded to the correct state (Fig. 5a and Supplementary Table 4).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    We have identified to our knowledge the shortest length of repeat that forms a beta-solenoid to date.

    “As an example, we have identified to our knowledge the shortest length of repeat that forms a beta-solenoid to date.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The AF2 database contains predictions of canonical protein isoforms for 21 model species, covering nearly every residue in 365,198 proteins.

    “The AF2 database has released predictions of the canonical protein isoforms for 21 model species, covering nearly every residue in 365,198 proteins.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [13] Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature 596, 590–596 (2021). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Previous findings indicate that core variant changes in particular lead to loss of stability, whereas surface variants are more likely to act through other mechanisms.

    “This is in line with previous findings that core variant changes in particular lead to loss of stability, whereas surface variants are more likely to act through other mechanisms30.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Høie, M. H., Cagiada, M., Beck Frederiksen, A. H., Stein, A. & Lindorff-Larsen, K. Predicting and interpreting large-scale mutagenesis data using analyses of protein stability and conservation. Cell Rep. 38, 110207 (2022). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The likely pathogenic variant p.R133W in DEGS1 is not predicted to affect stability, and hence likely has a different mechanism underlying disease.

    “The likely pathogenic variant p.R133W is not predicted to affect stability, and hence likely has a different mechanism underlying disease.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    Full protein models produced by AF2 have to be considered carefully given their potential errors, such as likely incorrect placement of low-confidence segments or low confidence in interdomain orientations.

    “However, the full protein models produced by AF2 have to be considered carefully given their potential errors, such as the likely incorrect placement of protein segments of low confidence or the low confidence in interdomain orientations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Contact predictions can be used to simultaneously fold and dock proteins, and AF2 can predict the structure of complexes.

    “it has been shown recently that contact predictions can be used to simultaneously fold and dock proteins17, and early reports have indicated that AF2 can predict the structure of complexes18–20, which it was not initially trained to handle.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [17] Pozzati, G. et al. Limits and potential of combined folding and docking. Bioinformatics 38, 954–961 (2021). source

      Not checked in the free run.

    • cites [18] Ko, J. & Lee, J. Can AlphaFold2 predict protein-peptide complex structures accurately? Preprint at bioRxiv https://doi.org/10.1101/2021.07.27.453972 (2021). source

      Not checked in the free run.

    • cites [19] Mirdita, M. et al. ColabFold: making protein folding accessible to all. Nat. Methods 19, 679–682 (2022). source

      Not checked in the free run.

    • cites [20] Bryant, P., Pozzati, G. & Elofsson, A. Improved prediction of protein-protein interactions using AlphaFold2. Nat. Commun. 13, 1265 (2022).

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Previous attempts to generate proteome-wide structural models include homology models such as SMR and modeling of Pfam domains using trRosetta.

    “previous attempts to generate ‘proteome-wide’ structural models include those based on homology models, such as the SWISS-MODEL Repository (SMR)14, and more recently, the modeling of known protein domains in the Pfam database15 using trRosetta16.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] Bienert, S. et al. The SWISS-MODEL Repository—new features and functionality. Nucleic Acids Res. 45, D313–D319 (2017). source

      Not checked in the free run.

    • cites [15] Mistry, J. et al. Pfam: The protein families database in 2021. Nucleic Acids Res. 49, D412–D419 (2021). source

      Not checked in the free run.

    • cites [16] Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. Proc. Natl Acad. Sci. USA 117, 1496–1503 (2020). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Around half the residues in AF2 predictions of the 11 model species are of low confidence, many of which may correspond to regions without a well-defined structure in isolation.

    “Around half the residues in AF2 predictions of the 11 model species are of low confidence, many of which may correspond to regions without a well-defined structure in isolation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    The platform provides metrics of uncertainty that reflect confidence in the structural assignment and uncertainty for pair-wise residue distances.

    “it is noteworthy that the platform provides metrics of uncertainty12 that have been shown to reflect confidence in the structural assignment—potentially linked to protein disorder—and uncertainty for pair-wise residue distances.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationown data

    The mean pLDDT of predicted pockets can be used as an additional criterion for pocket selection in AF2 structures.

    “Although there may be bias because high-confidence AF2 regions are more likely to have relevant deposited templates, we suggest that the mean pLDDT of predicted pockets can be used as an additional criterion for pocket selection in AF2 structures.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Two additional pathogenic variants in DEGS1 have ΔΔG values >1.5 kcal/mol, pointing towards loss of stability being the mechanism of pathogenicity; benign variants do not substantially affect protein stability.

    “Two additional pathogenic variants have ΔΔG values of >1.5 kcal/mol, pointing towards loss of stability being the mechanism of pathogenicity; the benign variants do not substantially affect protein stability, as expected (Fig. 3e).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The relation between disorder and predicted confidence could be a by-product of using the definition of disorder as regions not solved by X-ray crystallography, because AF2 is trained primarily on X-ray data.

    “However, we note that the protein disordered regions are often defined as regions that are not solved by X-ray crystallography. As AF2 is trained primarily on X-ray data, the relation between disorder and predicted confidence could be a by-product of using this definition of disorder.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: AF2 is trained primarily on X-ray data.
    • Rests on an assumption: Protein disordered regions are often defined as regions that are not solved by X-ray crystallography.

  • May combine two claims.

    Premisecitation

    The original 7OGG structure, solved using selenomethionine experimental phasing, had low resolution and anisotropy (ΔB = 80 Å2), with only 583 out of 850 total residues definitively modeled, 65 traced as unknown, and one peripheral 27-residue helix modeled out of register.

    “Originally solved using selenomethionine experimental phasing, the combination of low-resolution and anisotropy (ΔB = 80 Å2) meant that, although the core of the complex was confidently and correctly modeled, only 583 out of 850 total residues were definitively modeled by the authors, with a further 65 residues traced as unknown sequence and one peripheral 27-residue helix modeled out of register.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [50] Taschner, M. et al. Nse5/6 inhibits the Smc5/6 ATPase and modulates DNA substrate binding. EMBO J. 40, e107807 (2021). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    AF2 models, when their uncertainty is taken into account, can be applied to existing structural biology challenges, and their quality is near that of experimental models.

    “In summary, we find that AF2 models, when their uncertainty is taken into account, can be applied to existing structural biology challenges, and their quality is near that of experimental models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Co-evolutionary-information-based methods have been used to predict protein-protein interactions since the first development of direct coupling analysis algorithms.

    “Since the first development of direct coupling analysis algorithms, co-evolutionary-information-based methods have been used to predict protein-protein interactions40.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Weigt, M., White, R. A., Szurmant, H., Hoch, J. A. & Hwa, T. Identification of direct residue contacts in protein-protein interaction by message passing. Proc. Natl Acad. Sci. USA 106, 67–72 (2009). source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    The Nse5/6 complex structure 7OGG was not included in the AF2 training set, and no existing structures have ≥30% identity to either chain.

    “This was not included in the AF2 training set, and no existing structures have ≥30% identity to either chain.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [50] Taschner, M. et al. Nse5/6 inhibits the Smc5/6 ATPase and modulates DNA substrate binding. EMBO J. 40, e107807 (2021). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    These advances have led to the recent leap in performance demonstrated by Deepmind at CASP14.

    “These advances have led to the recent leap in performance demonstrated by Deepmind at CASP14 (ref. 12).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Several deep-learning-based methods, such as trRosetta and Raptor-X, can predict the structure of protein complexes.

    “It has been recently reported that several deep-learning-based methods, such as trRosetta16 and Raptor-X41, can predict the structure of protein complexes.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [16] Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. Proc. Natl Acad. Sci. USA 117, 1496–1503 (2020). source

      Not checked in the free run.

    • cites [41] Jing, X., Zeng, H., Wang, S. & Xu, J. A web-based protocol for interprotein contact prediction by deep learning. Methods Mol. Biol. 2074, 67–80 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Even for residues that can be modeled by distant homology, it is possible that AF2 models are more accurate, increasing their usefulness.

    “However, even for residues that can be modeled by distant homology, it is possible that the AF2 models are more accurate, increasing their usefulness.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Some reports have indicated that AF2 does not appear to be well suited to predict the structures of mutated proteins.

    “However, some reports have indicated that AF2 does not appear to be well suited to predict the structures of mutated proteins54,55.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [54] Pak, M. A. et al. Using AlphaFold to predict the impact of single mutations on protein stability and function. Preprint at bioRxiv https://doi.org/10.1101/2021.09.19.460937 (2021). source

      Not checked in the free run.

    • cites [55] Buel, G. R. & Walters, K. J. Can AlphaFold2 predict the impact of missense mutations on structure? Nat. Struct. Mol. Biol. 29, 1–2 (2022). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Both the AF2 method and a database of 365,198 protein models have been released.

    “Both the method and a database of 365,198 protein models have been released13”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature 596, 590–596 (2021). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    AF2 may add, on average, around 25% of confidently predicted residues to a given proteome, although this will vary depending on coverage by experimental and previous computational approaches.

    “We estimate that AF2 may add, on average, around 25% of confidently predicted residues to a given proteome, although this will vary depending on how much experimental and previous computational approaches have already covered.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    AF2 outcompetes standard docking approaches for prediction of complex structures while not requiring even starting protein structures.

    “Finally, we explored the application of AlphaFold2 for prediction of complex structures and found that it outcompetes standard docking approaches while not requiring even starting protein structures.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Regions of low-confidence AF2 predictions can be hypothesized to be disordered segments.

    “In addition, low AF2 prediction scores are enriched for protein disorder, suggesting that regions of low-confidence predictions can be hypothesized to be disordered segments.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    All analyses and results presented in the RoseTTAFold comparison are based on the subset of 17,006 protein sequences for which ENSEMBL and AlphaFold protein sequences were identical.

    “For comparison purposes, all the analyses and results presented here are based on the subset of 17,006 protein sequences for which the ENSEMBL and AlphaFold protein sequences were identical.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    For model building based on experimental data, there are examples of cases for which details were incorrect in regions where AF2 has high confidence, which underlies the need for detailed manual inspection.

    “For model building based on experimental data, we noted examples of cases for which details were incorrect in regions where AF2 has high confidence, which underlies the need for detailed manual inspection.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Examples that overlap with AF2's training set can be used to evaluate AF2's oligomeric state prediction capabilities, because AF2 was trained only on single chains.

    “Because AF2’s training was done only on single chains, we reasoned that examples, even if they overlap with the training set, could be used to evaluate AF2’s oligomeric state prediction capabilities.”
    Quote in the paperSubstantial claim
47 of 47

  • The paper relies on this without support.

    The implementation of computational protein structure prediction methods across structural-biology applications remains to be tested.

    “Although this has been independently verified, the implementation of these methods across structural-biology applications remains to be tested.”

  • The paper relies on this without support.

    Many questions remain regarding the extent to which AF2 approaches extend coverage of structural biology and the limitations of AF2 for applications in biology.

    “Yet many questions remain regarding the extent to which these approaches extend our coverage of structural biology, and the limitations of the AF2 method or structures derived from AF2 for applications in biology.”

  • The paper relies on this without support.

    A protein structure facilitates the generation of hypotheses regarding the impact of missense mutations.

    “A protein structure facilitates the generation of hypotheses regarding the impact of missense mutations.”

  • The paper relies on this without support.

    An agreement between the expected and observed impacts of mutations provides confidence in the accuracy of a structural model.

    “Conversely, an agreement between the expected and observed impacts of mutations provides confidence in the accuracy of a structural model.”

  • The paper relies on this without support.

    AF2 is trained primarily on X-ray data.

    “As AF2 is trained primarily on X-ray data, the relation between disorder and predicted confidence could be a by-product of using this definition of disorder.”

  • The paper relies on this without support.

    Protein disordered regions are often defined as regions that are not solved by X-ray crystallography.

    “However, we note that the protein disordered regions are often defined as regions that are not solved by X-ray crystallography.”

  • The paper relies on this without support.

    Many LBSp structures are from species not included in the public AlphaFold database.

    “In the latter case, we used locally modeled structures, as many LBSp structures are from species not included in the public database.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Trp A111 example cites the wrong figure panel

    Minor

    The text directs readers to Fig. 6f for Trp A111, but the figure caption places that example in panel e; panel f shows the rebuilt 7OGG structure.

  • Two PDB identifiers given for the same structural comparison

    Minor

    The metal-binding-center comparison identifies the reference structure as 4YZO, whereas the preceding description and figure caption identify it as 4ZYO.

  • Conflicting protein counts for the same stability-mutation dataset

    Minor

    The Results say the 2,648 stability mutations span 121 proteins, while the Methods say the same number of mutations spans 132 proteins. The text does not describe a different subset that would account for the discrepancy.

3 of 3

Cites

Flagged first, then by citations

5 of the 91 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Mirdita, M. et al. ColabFold—making protein folding accessible to all. Nat. Methods 19, 679–682 (2022). · DOI · Paper page

Cited by

Sorted by citations

Related

Authors
Akdel, M., Pires, D. E. V., Pardo, E. P., Jänes, J., Zalevsky, A. O., Mészáros, B., Bryant, P., Good, L. L., Laskowski, R. A., Pozzati, G., Shenoy, A., Zhu, W., Kundrotas, P., Serra, V. R., Rodrigues, C. H. M., Dunham, A. S., Burke, D., Borkakoti, N., Velankar, S., Frost, A., Basquin, J., Lindorff-Larsen, K., Bateman, A., Kajava, A. V., Valencia, A., Ovchinnikov, S., Durairaj, J., Ascher, D. B., Thornton, J. M., Davey, N. E., Stein, A., Elofsson, A., Croll, T. I., & Beltrao, P.
Affiliations
  • Wageningen University & Research (NL)
  • The University of Melbourne (AU)
  • European Bioinformatics Institute (GB)
  • Russian Academy of Sciences (RU)
  • European Molecular Biology Laboratory (DE)
  • Science for Life Laboratory (SE)
  • University of Copenhagen (DK)
  • University of California, San Francisco (US)
  • Max Planck Institute of Biochemistry (DE)
  • Centre National de la Recherche Scientifique (FR)
  • Barcelona Supercomputing Center (ES)
  • Harvard University (US)
  • University of Basel (CH)
  • The University of Queensland (AU)
  • Institute of Cancer Research (GB)
  • University of Cambridge (GB)
Corresponding author
  • Valencia, Alfonso
  • Ovchinnikov, Sergey
  • Durairaj, Janani
  • Ascher, David B.
  • Thornton, Janet M.
  • Davey, Norman E.
  • Stein, Amelie
  • Elofsson, Arne
  • Croll, Tristan I.
ORCID
Venue
Nature Structural & Molecular Biology
Publisher
Springer Science and Business Media LLC
Published
2022-11
Volume
29
Issue
11
Pages
1056-1067
ISSN
  • 1545-9985 (ISSN-L)
  • 1545-9993
DOI
10.1038/s41594-022-00849-w
PMC
PMC9663297
OpenAlex
W4308463927
Semantic Scholar
7bde0455b85732e9853f17f6b671c1e91884af78
Cited by
675
References
91
Citations by year
2026: 137 · 2025: 143 · 2024: 171 · 2023: 174 · 2022: 47 · 2021: 2 · 2012: 1
FWCI
44.48
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Structural & Molecular Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
  • edoc (University of Basel) (acceptedVersion)
  • HAL (submittedVersion, other-oa)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • Socio-Environmental Systems Modeling (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Funders
  • National Science Foundation (2032259)
  • Simons Foundation (735929LPI)
  • Wellcome Trust
  • Swedish e-Science Research Centre
  • Cancer Research UK (28159)
  • European Commission (HORIZON2020, 842490)
  • Eidgenössische Technische Hochschule Zürich
  • Lundbeckfonden (R272-2017-4528)
  • Novo Nordisk
  • Vetenskapsrådet (2016-06301, Berzelius-2021-29)
  • State Government of Victoria
  • Russian Science Foundation (20-14-00121)
  • Novo Nordisk Fonden (NNF18OC0033950)
  • ETH Zürich Foundation
  • H. Lundbeck A/S
  • Helmut Horten Stiftung
  • National Institutes of Health (RYC2019-026415-I, DP5OD026389)
  • Medical Research Council
  • National Health and Medical Research Council (1174405)
  • National Institute of Allergy and Infectious Diseases (R21AI156595)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
  • Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
Keywords
AlphaFold2, protein structure prediction, protein disorder prediction, protein complex modeling, ligand binding site prediction, confidence measures, Protein Data Bank
MeSH terms
Binding Sites, Furylfuramide, Protein Conformation, Proteins, Computational Biology, Databases, Protein

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review