CoChat

Blind assessment of monomeric AlphaFold2 protein structure models with experimental NMR data

Open access
Li, Ethan H., Spaman, Laura E., Tejero, Roberto, Janet Huang, Yuanpeng, Ramelot, Theresa A., Fraga, Keith J., Prestegard, James H., Kennedy, Michael A., Montelione, Gaetano T. · 2023 · Journal of Magnetic Resonance
Cited by 32

The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.

Abstract

Recent advances in molecular modeling of protein structures are changing the field of structural biology. AlphaFold-2 (AF2), an AI system developed by DeepMind, Inc., utilizes attention-based deep learning to predict models of protein structures with high accuracy relative to structures determined by X-ray crystallography and cryo-electron microscopy (cryoEM). Comparing AF2 models to structures determined using solution NMR data, both high similarities and distinct differences have been observed. Since AF2 was trained on X-ray crystal and cryoEM structures, we assessed how accurately AF2 can model small, monomeric, solution protein NMR structures which (i) were not used in the AF2 training data set, and (ii) did not have homologous structures in the Protein Data Bank at the time of AF2 training. We identified nine open source protein NMR data sets for such “blind” targets, including chemical shift, raw NMR FID data, NOESY peak lists, and (for 1 case) 15N-1H residual dipolar coupling data. For these nine small (70–108 residues) monomeric proteins, we generated AF2 prediction models and assessed how well these models fit to these experimental NMR data, using several well-established NMR structure validation tools. In most of these cases, the AF2 models fit the NMR data nearly as well, or sometimes better than, the corresponding NMR structure models previously deposited in the Protein Data Bank. These results provide benchmark NMR data for assessing new NMR data analysis and protein structure prediction methods. They also document the potential for using AF2 as a guiding tool in protein NMR data analysis, and more generally for hypothesis generation in structural biology research.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jmr.2023.107481

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 20 references

Research works printed but not linked: 46

  1. Jones, 2022, The impact of AlphaFold2 one year on, Nat. Methods, 19, 15, 10.1038/s41592-021-01365-3 · DOI · Paper page
  2. Jumper, 2021, Applying and improving AlphaFold at CASP14, Proteins, 89, 1711, 10.1002/prot.26257 · DOI · Paper page
  3. Huang, 2021, Assessment of prediction methods for protein structures determined by NMR in CASP14: Impact of AlphaFold2, Proteins, 89, 1959, 10.1002/prot.26246 · DOI · Paper page
  4. T.C. Terwilliger, D. Liebschner, T.I. Croll, C.J. Williams, A.J. McCoy, B.K. Poon, P.V. Afonine, R.D. Oeffner, J.S. Richardson, R.J. Read, P.D. Adams, AlphaFold predictions: great hypotheses but no match for experiment, bioRxiv, (2022) 2022.2011.2021.517405.
  5. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Long Beach, California, USA, 2017, pp. 6000–6010.
  6. Kamisetty, 2013, Assessing the utility of coevolution-based residue-residue contact predictions in a sequence- and structure-rich era, Proc. Natl. Acad. Sci. USA, 110, 15674, 10.1073/pnas.1314045110 · DOI · Paper page
  7. Burley, 2021, Open-access data: A cornerstone for artificial intelligence approaches to protein structure prediction, Structure, 29, 515, 10.1016/j.str.2021.04.010 · DOI · Paper page
  8. Sala, 2019, Protein structure prediction assisted with sparse NMR data in CASP13, Proteins, 87, 1315, 10.1002/prot.25837 · DOI · Paper page
  9. Khaje, 2022, Validated determination of NRG1 Ig-like domain structure by mass spectrometry coupled with computational modeling, Commun. Biol., 5, 452, 10.1038/s42003-022-03411-y · doi:10.1038/s42003-022-03411-y
  10. Ulrich, 2008, BioMagResBank, Nucl. Acids Res., 36, D402, 10.1093/nar/gkm957 · DOI · Paper page
  11. Klukowski, 2022, Rapid protein assignments and structures from raw NMR spectra with the deep learning technique ARTINA, Nat. Commun., 13, 6151, 10.1038/s41467-022-33879-5 · DOI · Paper page
  12. Chen, 2010, MolProbity: all-atom structure validation for macromolecular crystallography, Acta Crystallogr. Sect. D Biol. Crystallogr., 66, 12, 10.1107/S0907444909042073 · DOI · Paper page
  13. Bhattacharya, 2007, Evaluating protein structures determined by structural genomics consortia, Proteins, 66, 778, 10.1002/prot.21165 · DOI · Paper page
  14. Tejero, 2013, PDBStat: a universal restraint converter and restraint analysis software package for protein NMR, J. Biomol. NMR, 56, 337, 10.1007/s10858-013-9753-7 · DOI · Paper page
  15. Huang, 2005, Protein NMR recall, precision, and F-measure scores (RPF scores): structure quality assessment measures based on information retrieval statistics, J. Am. Chem. Soc., 127, 1665, 10.1021/ja047109h · DOI · Paper page
  16. Huang, 2012, RPF: a quality assessment tool for protein NMR structures, Nucl. Acids Res., 40, W542, 10.1093/nar/gks373 · DOI · Paper page
  17. Kirchner, 2011, Objective identification of residue ranges for the superposition of protein structures, BMC Bioinf., 12, 170, 10.1186/1471-2105-12-170 · DOI · Paper page
  18. Snyder, 2014, The expanded FindCore method for identification of a core atom set for assessment of protein structure prediction, Proteins, 82, 219, 10.1002/prot.24490 · DOI
  19. Snyder, 2005, Clustering algorithms for identifying core atom sets and for assessing the precision of protein structure ensembles, Proteins, 59, 673, 10.1002/prot.20402 · DOI
  20. Rosato, 2012, Blind testing of routine, fully automated determination of protein structures from NMR data, Structure, 20, 227, 10.1016/j.str.2012.01.002 · DOI
  21. Losonczi, 1999, Order matrix analysis of residual dipolar couplings using singular value decomposition, J. Magn. Reson., 138, 334, 10.1006/jmre.1999.1754 · DOI · Paper page
  22. Cornilescu, 1998, Validation of protein structure from anisotropic carbonyl chemical shifts in a dilute liquid crystalline phase, J. Amer. Chem. Soc., 120, 6836, 10.1021/ja9812610 · DOI · Paper page
  23. Clore, 1999, R-factor, free R, and complete cross-validation for dipolar coupling refinement of NMR structures, J. Amer. Chem. Soc., 121, 9008, 10.1021/ja991789k · DOI · Paper page
  24. Montelione, 2013, Recommendations of the wwPDB NMR Validation Task Force, Structure, 21, 1563, 10.1016/j.str.2013.07.021 · DOI · Paper page
  25. Zhang, 2004, Scoring function for automated assessment of protein structure template quality, Prot.: Struct. Funct., Bioinform., 57, 702
  26. Lee, 2010, Three-dimensional structure of the weakly associated protein homodimer SeR13 using RDCs and paramagnetic surface mapping, Prot. Sci., 19, 1673, 10.1002/pro.447 · DOI
  27. DeLano, 2002, The PyMOL Molecular Graphics System, Version 1.5.0.4 Schrödinger, LLC., The PyMOL Molecular Graphics System, Version 1.5.0.4 Schrödinger, LLC.
  28. Montelione, 2012, The protein structure initiative: achievements and visions for the future, F1000 Biol. Rep., 4, 7, 10.3410/B4-7 · DOI
  29. Lee, 2015, NMRFAM-SPARKY: enhanced software for biomolecular NMR spectroscopy, Bioinformatics, 31, 1325, 10.1093/bioinformatics/btu830 · DOI · Paper page
  30. Bartels, 1995, The program XEASY for computer-supported NMR spectral analysis of biological macromolecules, J. Biomol. NMR, 6, 1, 10.1007/BF00417486 · DOI · Paper page
  31. Hyberts, 1992, The solution structure of eglin c based on measurements of many NOEs and coupling constants and its comparison with X-ray structures, Prot. Sci., 1, 736, 10.1002/pro.5560010606 · DOI · Paper page
  32. Laskowski, 1993, PROCHECK - a program to check the stereochemical quality of protein structures, J. Appl. Crystallogr., 26, 283, 10.1107/S0021889892009944 · DOI · Paper page
  33. Lovell, 2003, Structure validation by Calpha geometry: phi, psi and Cbeta deviation, Proteins, 50, 437, 10.1002/prot.10286 · DOI · Paper page
  34. Rosato, 2009, CASD-NMR: critical assessment of automated structure determination by NMR, Nat. Methods, 6, 625, 10.1038/nmeth0909-625 · DOI
  35. Takatsu, 2022, Biophysical analysis of Gaussia Luciferase bioluminescence mechanisms using a non-oxidizable coelenterazine, BBA, Advances
  36. Robertson, 2021, Concordance of X-ray and AlphaFold2 Models of SARS-CoV-2 Main Protease with Residual Dipolar Couplings Measured in Solution, J. Am. Chem. Soc., 143, 19306, 10.1021/jacs.1c10588 · DOI · Paper page
  37. Chiliveri, 2022, Advances in NMR spectroscopy of weakly aligned biomolecular systems, Chem. Rev., 122, 9307, 10.1021/acs.chemrev.1c00730 · DOI · Paper page
  38. Williamson, 1985, Solution conformation of proteinase inhibitor IIA from bull seminal plasma by 1H nuclear magnetic resonance and distance geometry, J. Mol. Biol., 182, 295, 10.1016/0022-2836(85)90347-X · DOI · Paper page
  39. Wagner, 1987, Protein structures in solution by nuclear magnetic resonance and distance geometry. The polypeptide fold of the basic pancreatic trypsin inhibitor determined using two different algorithms. DISGEO and DISMAN, J. Mol. Biol., 196, 611, 10.1016/0022-2836(87)90037-4 · DOI
  40. Wagner, 1987, Spatial structure of rabbit liver metallothionein-2 in solution by NMR, Experientia Suppl., 52, 149, 10.1007/978-3-0348-6784-9_8 · doi:10.1007/978-3-0348-6784-9_8
  41. Montelione, 1987, Solution structure of murine epidermal growth factor: determination of the polypeptide backbone chain-fold by nuclear magnetic resonance and distance geometry, Proc. Natl. Acad. Sci. USA, 84, 5226, 10.1073/pnas.84.15.5226 · DOI · Paper page
  42. H.K. Wayment-Steele, S. Ovchinnikov, L. Colwell, D. Kern, Prediction of multiple conformational states by combining sequence clustering with AlphaFold2, bioRxiv, (2022) 2022.2010.2017.512570. · DOI · Paper page
  43. Tejero, 1996, Simulated annealing with restrained molecular dynamics using CONGEN: energy refinement of the NMR solution structures of epidermal and type-alpha transforming growth factors, Prot. Sci., 5, 578, 10.1002/pro.5560050403 · doi:10.1002/pro.5560050403
  44. Gibbs, 2018, Inhibitor bound dengue NS2B-NS3pro reveals multiple dynamic binding modes, Biochemistry, 57, 1591, 10.1021/acs.biochem.7b01127 · DOI
  45. Koepnick, 2019, De novo protein design by citizen scientists, Nature, 570, 390, 10.1038/s41586-019-1274-4 · DOI · Paper page
  46. Yang, 2020, Improved protein structure prediction using predicted interresidue orientations, Proc. Natl. Acad. Sci. USA, 117, 1496, 10.1073/pnas.1914677117 · DOI · Paper page

Cited by

See all 32

Metadata

Authors
Li, E. H., Spaman, L. E., Tejero, R., Janet Huang, Y., Ramelot, T. A., Fraga, K. J., Prestegard, J. H., Kennedy, M. A., & Montelione, G. T.
Venue
Journal of Magnetic Resonance
Publisher
Elsevier BV
Published
2023-07
Volume
352
Pages
107481
ISSN
1090-7807 (ISSN-L)
OpenAlex
W4377089153
Cited by
32
References
20
FWCI
2.39
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
National Institutes of Health (R35-GM141818)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Blind assessment of monomeric AlphaFold2 protein structure models with experimental NMR data. https://app.cochat.ai/papers/10.1016/j.jmr.2023.107481
The paper
Li, E. H., Spaman, L. E., Tejero, R., Janet Huang, Y., Ramelot, T. A., Fraga, K. J., Prestegard, J. H., Kennedy, M. A., & Montelione, G. T. (2023). Blind assessment of monomeric AlphaFold2 protein structure models with experimental NMR data. Journal of Magnetic Resonance. https://doi.org/10.1016/j.jmr.2023.107481

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.