Blind assessment of monomeric AlphaFold2 protein structure models with experimental NMR data
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Recent advances in molecular modeling of protein structures are changing the field of structural biology. AlphaFold-2 (AF2), an AI system developed by DeepMind, Inc., utilizes attention-based deep learning to predict models of protein structures with high accuracy relative to structures determined by X-ray crystallography and cryo-electron microscopy (cryoEM). Comparing AF2 models to structures determined using solution NMR data, both high similarities and distinct differences have been observed. Since AF2 was trained on X-ray crystal and cryoEM structures, we assessed how accurately AF2 can model small, monomeric, solution protein NMR structures which (i) were not used in the AF2 training data set, and (ii) did not have homologous structures in the Protein Data Bank at the time of AF2 training. We identified nine open source protein NMR data sets for such “blind” targets, including chemical shift, raw NMR FID data, NOESY peak lists, and (for 1 case) 15N-1H residual dipolar coupling data. For these nine small (70–108 residues) monomeric proteins, we generated AF2 prediction models and assessed how well these models fit to these experimental NMR data, using several well-established NMR structure validation tools. In most of these cases, the AF2 models fit the NMR data nearly as well, or sometimes better than, the corresponding NMR structure models previously deposited in the Protein Data Bank. These results provide benchmark NMR data for assessing new NMR data analysis and protein structure prediction methods. They also document the potential for using AF2 as a guiding tool in protein NMR data analysis, and more generally for hypothesis generation in structural biology research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434
Protein 3D Structure Computed from Evolutionary Sequence Variation
Marks, D. S., Colwell, L. J., Sheridan, R., Hopf, T. A., Pagnani, A., Zecchina, R., & Sander, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028766 · Cited by 1,144
LGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069
De novo protein design by deep network hallucination
Anishchenko, I., Chidyausiku, T. M., Ovchinnikov, S., Pellock, S. J., & Baker, D. (2020) · doi:10.1101/2020.07.22.211482 · Cited by 629
Ovchinnikov, S., Kamisetty, H., & Baker, D. (2014) · eLife · doi:10.7554/elife.02030 · Cited by 629
Protein structure prediction from sequence variation
Marks, D. S., Hopf, T. A., & Sander, C. (2012) · Nature Biotechnology · doi:10.1038/nbt.2419 · Cited by 622
Research works printed but not linked: 46
- Jones, 2022, The impact of AlphaFold2 one year on, Nat. Methods, 19, 15, 10.1038/s41592-021-01365-3 · DOI · Paper page
- Jumper, 2021, Applying and improving AlphaFold at CASP14, Proteins, 89, 1711, 10.1002/prot.26257 · DOI · Paper page
- Huang, 2021, Assessment of prediction methods for protein structures determined by NMR in CASP14: Impact of AlphaFold2, Proteins, 89, 1959, 10.1002/prot.26246 · DOI · Paper page
- T.C. Terwilliger, D. Liebschner, T.I. Croll, C.J. Williams, A.J. McCoy, B.K. Poon, P.V. Afonine, R.D. Oeffner, J.S. Richardson, R.J. Read, P.D. Adams, AlphaFold predictions: great hypotheses but no match for experiment, bioRxiv, (2022) 2022.2011.2021.517405.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Long Beach, California, USA, 2017, pp. 6000–6010.
- Kamisetty, 2013, Assessing the utility of coevolution-based residue-residue contact predictions in a sequence- and structure-rich era, Proc. Natl. Acad. Sci. USA, 110, 15674, 10.1073/pnas.1314045110 · DOI · Paper page
- Burley, 2021, Open-access data: A cornerstone for artificial intelligence approaches to protein structure prediction, Structure, 29, 515, 10.1016/j.str.2021.04.010 · DOI · Paper page
- Sala, 2019, Protein structure prediction assisted with sparse NMR data in CASP13, Proteins, 87, 1315, 10.1002/prot.25837 · DOI · Paper page
- Khaje, 2022, Validated determination of NRG1 Ig-like domain structure by mass spectrometry coupled with computational modeling, Commun. Biol., 5, 452, 10.1038/s42003-022-03411-y · doi:10.1038/s42003-022-03411-y
- Ulrich, 2008, BioMagResBank, Nucl. Acids Res., 36, D402, 10.1093/nar/gkm957 · DOI · Paper page
- Klukowski, 2022, Rapid protein assignments and structures from raw NMR spectra with the deep learning technique ARTINA, Nat. Commun., 13, 6151, 10.1038/s41467-022-33879-5 · DOI · Paper page
- Chen, 2010, MolProbity: all-atom structure validation for macromolecular crystallography, Acta Crystallogr. Sect. D Biol. Crystallogr., 66, 12, 10.1107/S0907444909042073 · DOI · Paper page
- Bhattacharya, 2007, Evaluating protein structures determined by structural genomics consortia, Proteins, 66, 778, 10.1002/prot.21165 · DOI · Paper page
- Tejero, 2013, PDBStat: a universal restraint converter and restraint analysis software package for protein NMR, J. Biomol. NMR, 56, 337, 10.1007/s10858-013-9753-7 · DOI · Paper page
- Huang, 2005, Protein NMR recall, precision, and F-measure scores (RPF scores): structure quality assessment measures based on information retrieval statistics, J. Am. Chem. Soc., 127, 1665, 10.1021/ja047109h · DOI · Paper page
- Huang, 2012, RPF: a quality assessment tool for protein NMR structures, Nucl. Acids Res., 40, W542, 10.1093/nar/gks373 · DOI · Paper page
- Kirchner, 2011, Objective identification of residue ranges for the superposition of protein structures, BMC Bioinf., 12, 170, 10.1186/1471-2105-12-170 · DOI · Paper page
- Snyder, 2014, The expanded FindCore method for identification of a core atom set for assessment of protein structure prediction, Proteins, 82, 219, 10.1002/prot.24490 · DOI
- Snyder, 2005, Clustering algorithms for identifying core atom sets and for assessing the precision of protein structure ensembles, Proteins, 59, 673, 10.1002/prot.20402 · DOI
- Rosato, 2012, Blind testing of routine, fully automated determination of protein structures from NMR data, Structure, 20, 227, 10.1016/j.str.2012.01.002 · DOI
- Losonczi, 1999, Order matrix analysis of residual dipolar couplings using singular value decomposition, J. Magn. Reson., 138, 334, 10.1006/jmre.1999.1754 · DOI · Paper page
- Cornilescu, 1998, Validation of protein structure from anisotropic carbonyl chemical shifts in a dilute liquid crystalline phase, J. Amer. Chem. Soc., 120, 6836, 10.1021/ja9812610 · DOI · Paper page
- Clore, 1999, R-factor, free R, and complete cross-validation for dipolar coupling refinement of NMR structures, J. Amer. Chem. Soc., 121, 9008, 10.1021/ja991789k · DOI · Paper page
- Montelione, 2013, Recommendations of the wwPDB NMR Validation Task Force, Structure, 21, 1563, 10.1016/j.str.2013.07.021 · DOI · Paper page
- Zhang, 2004, Scoring function for automated assessment of protein structure template quality, Prot.: Struct. Funct., Bioinform., 57, 702
- Lee, 2010, Three-dimensional structure of the weakly associated protein homodimer SeR13 using RDCs and paramagnetic surface mapping, Prot. Sci., 19, 1673, 10.1002/pro.447 · DOI
- DeLano, 2002, The PyMOL Molecular Graphics System, Version 1.5.0.4 Schrödinger, LLC., The PyMOL Molecular Graphics System, Version 1.5.0.4 Schrödinger, LLC.
- Montelione, 2012, The protein structure initiative: achievements and visions for the future, F1000 Biol. Rep., 4, 7, 10.3410/B4-7 · DOI
- Lee, 2015, NMRFAM-SPARKY: enhanced software for biomolecular NMR spectroscopy, Bioinformatics, 31, 1325, 10.1093/bioinformatics/btu830 · DOI · Paper page
- Bartels, 1995, The program XEASY for computer-supported NMR spectral analysis of biological macromolecules, J. Biomol. NMR, 6, 1, 10.1007/BF00417486 · DOI · Paper page
- Hyberts, 1992, The solution structure of eglin c based on measurements of many NOEs and coupling constants and its comparison with X-ray structures, Prot. Sci., 1, 736, 10.1002/pro.5560010606 · DOI · Paper page
- Laskowski, 1993, PROCHECK - a program to check the stereochemical quality of protein structures, J. Appl. Crystallogr., 26, 283, 10.1107/S0021889892009944 · DOI · Paper page
- Lovell, 2003, Structure validation by Calpha geometry: phi, psi and Cbeta deviation, Proteins, 50, 437, 10.1002/prot.10286 · DOI · Paper page
- Rosato, 2009, CASD-NMR: critical assessment of automated structure determination by NMR, Nat. Methods, 6, 625, 10.1038/nmeth0909-625 · DOI
- Takatsu, 2022, Biophysical analysis of Gaussia Luciferase bioluminescence mechanisms using a non-oxidizable coelenterazine, BBA, Advances
- Robertson, 2021, Concordance of X-ray and AlphaFold2 Models of SARS-CoV-2 Main Protease with Residual Dipolar Couplings Measured in Solution, J. Am. Chem. Soc., 143, 19306, 10.1021/jacs.1c10588 · DOI · Paper page
- Chiliveri, 2022, Advances in NMR spectroscopy of weakly aligned biomolecular systems, Chem. Rev., 122, 9307, 10.1021/acs.chemrev.1c00730 · DOI · Paper page
- Williamson, 1985, Solution conformation of proteinase inhibitor IIA from bull seminal plasma by 1H nuclear magnetic resonance and distance geometry, J. Mol. Biol., 182, 295, 10.1016/0022-2836(85)90347-X · DOI · Paper page
- Wagner, 1987, Protein structures in solution by nuclear magnetic resonance and distance geometry. The polypeptide fold of the basic pancreatic trypsin inhibitor determined using two different algorithms. DISGEO and DISMAN, J. Mol. Biol., 196, 611, 10.1016/0022-2836(87)90037-4 · DOI
- Wagner, 1987, Spatial structure of rabbit liver metallothionein-2 in solution by NMR, Experientia Suppl., 52, 149, 10.1007/978-3-0348-6784-9_8 · doi:10.1007/978-3-0348-6784-9_8
- Montelione, 1987, Solution structure of murine epidermal growth factor: determination of the polypeptide backbone chain-fold by nuclear magnetic resonance and distance geometry, Proc. Natl. Acad. Sci. USA, 84, 5226, 10.1073/pnas.84.15.5226 · DOI · Paper page
- H.K. Wayment-Steele, S. Ovchinnikov, L. Colwell, D. Kern, Prediction of multiple conformational states by combining sequence clustering with AlphaFold2, bioRxiv, (2022) 2022.2010.2017.512570. · DOI · Paper page
- Tejero, 1996, Simulated annealing with restrained molecular dynamics using CONGEN: energy refinement of the NMR solution structures of epidermal and type-alpha transforming growth factors, Prot. Sci., 5, 578, 10.1002/pro.5560050403 · doi:10.1002/pro.5560050403
- Gibbs, 2018, Inhibitor bound dengue NS2B-NS3pro reveals multiple dynamic binding modes, Biochemistry, 57, 1591, 10.1021/acs.biochem.7b01127 · DOI
- Koepnick, 2019, De novo protein design by citizen scientists, Nature, 570, 390, 10.1038/s41586-019-1274-4 · DOI · Paper page
- Yang, 2020, Improved protein structure prediction using predicted interresidue orientations, Proc. Natl. Acad. Sci. USA, 117, 1496, 10.1073/pnas.1914677117 · DOI · Paper page
Cited by
The power and pitfalls of AlphaFold2 for structure prediction beyond rigid globular proteins
Agarwal, V., & McShan, A. C. (2024) · Nature Chemical Biology · doi:10.1038/s41589-024-01638-w · Cited by 155
The pathogen-encoded signalling receptor Tir exploits host-like intrinsic disorder for infection
Vieira, M. F. M., Hernandez, G., Zhong, Q., Arbesú, M., Veloso, T., Gomes, T., Martins, M. L., Monteiro, H., Frazão, C., Frankel, G., et al. (2024) · Communications Biology · doi:10.1038/s42003-024-05856-9 · Cited by 14
Gavalda-Garcia, J., Dixit, B., Díaz, A., Ghysels, A., & Vranken, W. (2025) · Journal of Molecular Biology · doi:10.1016/j.jmb.2024.168900 · Cited by 12
Assisting and accelerating NMR assignment with restrained structure prediction
Liu, S., Chu, H., Xie, Y., Wu, F., Mu, F., Wei, J., Ni, N., Wang, C., Zhang, J., Chen, M., et al. (2025) · Communications Biology · doi:10.1038/s42003-025-08466-1 · Cited by 10
Zeinalilathori, S., Russell, R. W., Ross, J. L., Modla, S., Caplan, J. L., Polenova, T., & Thorpe, C. (2026) · Protein Science · doi:10.1002/pro.70431 · Cited by 1
Metadata
- Authors
- Li, E. H., Spaman, L. E., Tejero, R., Janet Huang, Y., Ramelot, T. A., Fraga, K. J., Prestegard, J. H., Kennedy, M. A., & Montelione, G. T.
- Venue
- Journal of Magnetic Resonance
- Publisher
- Elsevier BV
- Published
- 2023-07
- Volume
- 352
- Pages
- 107481
- ISSN
- 1090-7807 (ISSN-L)
- OpenAlex
- W4377089153
- Cited by
- 32
- References
- 20
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Funders
- National Institutes of Health (R35-GM141818)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Blind assessment of monomeric AlphaFold2 protein structure models with experimental NMR data. https://app.cochat.ai/papers/10.1016/j.jmr.2023.107481
- The paper
- Li, E. H., Spaman, L. E., Tejero, R., Janet Huang, Y., Ramelot, T. A., Fraga, K. J., Prestegard, J. H., Kennedy, M. A., & Montelione, G. T. (2023). Blind assessment of monomeric AlphaFold2 protein structure models with experimental NMR data. Journal of Magnetic Resonance. https://doi.org/10.1016/j.jmr.2023.107481
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.