Assessment of prediction methods for protein structures determined by NMR in CASP14 : Impact of AlphaFold2
Open accessAbstract
NMR studies can provide unique information about protein conformations in solution. In CASP14, three reference structures provided by solution NMR methods were available (T1027, T1029, and T1055), as well as a fourth data set of NMR-derived contacts for an integral membrane protein (T1088). For the three targets with NMR-based structures, the best prediction results ranged from very good (GDT_TS = 0.90, for T1055) to poor (GDT_TS = 0.47, for T1029). We explored the basis of these results by comparing all CASP14 prediction models against experimental NMR data. For T1027, NMR data reveal extensive internal dynamics, presenting a unique challenge for protein structure prediction methods. The analysis of T1029 motivated exploration of a novel method of "inverse structure determination," in which an AlphaFold2 model was used to guide NMR data analysis. NMR data provided to CASP predictor groups for target T1088, a 238-residue integral membrane porin, was also used to assess several NMR-assisted prediction methods. Most groups involved in this exercise generated similar beta-barrel models, with good agreement with the experimental data. However, as was also observed in CASP13, some pure prediction groups that did not use any NMR data generated models for T1088 that better fit the NMR data than the models generated using these experimental data. These results demonstrate the remarkable power of modern methods to predict structures of proteins with accuracies rivaling solution NMR structures, and that it is now possible to reliably use prediction models to guide and complement experimental NMR data analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AF2 has been demonstrated to fail in predicting multiple structures of metamorphic proteins, proteins with apo/holo conformational changes and other multi-state proteins using its default settings.
“yet AF2 has been demonstrated to fail in predicting multiple structures of metamorphic proteins11, proteins with apo/holo conformational changes12 and other multi-state proteins13 using its default settings.”
From Predicting multiple conformations via sequence clustering and AlphaFold2 (2024) · cites this paper as [13]
For the other NMR target, the best computed structures agree better with the experimental NOE data than does the experimental structure.
“For the other, the best computed structures agree better with the experimental NOE data than does the experimental structure.”
From Critical assessment of methods of protein structure prediction (CASP)—Round <scp>XIV</scp> (2021) · cites this paper as [19]
Analysis by Gaetano Montelione's group shows that one NMR target, T1027, is a dynamic structure and the best computed structures may correspond to a member of the ensemble.
“Analysis by Gaetano Montelione’s group19, shows that one of these, T1027, is a dynamic structure and the best computed structures may correspond to a member of the ensemble.”
From Critical assessment of methods of protein structure prediction (CASP)—Round <scp>XIV</scp> (2021) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
PROCHECK: a program to check the stereochemical quality of protein structures
Laskowski, R. A., MacArthur, M. W., Moss, D. S., & Thornton, J. M. (1993) · Journal of Applied Crystallography · doi:10.1107/s0021889892009944 · Cited by 24,581 · see citing works
<i>MolProbity</i> : all-atom structure validation for macromolecular crystallography
Chen, V. B., Arendall, W. B., Headd, J. J., Keedy, D. A., Immormino, R. M., Kapral, G. J., Murray, L. W., Richardson, J. S., & Richardson, D. C. (2010) · Acta Crystallographica Section D Biological Crystallography · doi:10.1107/s0907444909042073 · Cited by 14,115 · see citing works
Statistical mechanics of chain molecules
Flory, P. J., & Volkenstein, M. V. (1969) · Biopolymers · doi:10.1002/bip.1969.360080514 · Cited by 6,449 · see citing works
Accurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,542 · see citing works
Structure validation by Cα geometry: ϕ,ψ and Cβ deviation
Lovell, S. C., Davis, I. W., Arendall, W. B., de Bakker, P. I. W., Word, J. M., Prisant, M. G., Richardson, J. S., & Richardson, D. C. (2003) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.10286 · Cited by 4,595 · see citing works
MolProbity: all-atom contacts and structure validation for proteins and nucleic acids
Davis, I., Leaver‐Fay, A., Chen, V. B., Block, J. N., Kapral, G. J., Wang, X., Murray, L. W., Arendall, W. B., Snoeyink, J. S., Richardson, J. S., et al. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm216 · Cited by 3,915 · see citing works
Assessment of protein models with three-dimensional profiles
Lüthy, R., Bowie, J., & Eisenberg, D. (1992) · Nature · doi:10.1038/356083a0 · Cited by 3,588 · see citing works
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045 · see citing works
Recognition of errors in three‐dimensional structures of proteins
Sippl, M. J. (1993) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.340170404 · Cited by 2,156 · see citing works
Morcos, F., Pagnani, A., Lunt, B., Bertolino, A., Marks, D. S., Sander, C., Zecchina, R., Onuchic, J. N., Hwa, T., & Weigt, M. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1111471108 · Cited by 1,434 · see citing works
Version 1.2 of the Crystallography and NMR system
Brünger, A. T. (2007) · Nature Protocols · doi:10.1038/nprot.2007.406 · Cited by 1,335 · see citing works
Protein 3D Structure Computed from Evolutionary Sequence Variation
Marks, D. S., Colwell, L. J., Sheridan, R., Hopf, T. A., Pagnani, A., Zecchina, R., & Sander, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028766 · Cited by 1,144 · see citing works
LGA: a method for finding 3D similarities in protein structures
Zemla, A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg571 · Cited by 1,069 · see citing works
Using Xplor–NIH for NMR molecular structure determination
Schwieters, C. D., Kuszewski, J. J., & Clore, G. M. (2005) · Progress in Nuclear Magnetic Resonance Spectroscopy · doi:10.1016/j.pnmrs.2005.10.001 · Cited by 795 · see citing works
Evaluating protein structures determined by structural genomics consortia
Bhattacharya, A., Tejero, R., & Montelione, G. T. (2006) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.21165 · Cited by 729 · see citing works
Self-citationProtein structure prediction from sequence variation
Marks, D. S., Hopf, T. A., & Sander, C. (2012) · Nature Biotechnology · doi:10.1038/nbt.2419 · Cited by 622 · see citing works
ARIA2: Automated NOE assignment and data integration in NMR structure calculation
Rieping, W., Habeck, M., Bardiaux, B., Bernard, A., Malliavin, T. E., & Nilges, M. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl589 · Cited by 542 · see citing works
Critical assessment of methods of protein structure prediction (CASP)—Round <scp>XIV</scp>
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26237 · Cited by 503 · see citing works
Self-citationCombined automated NOE assignment and structure calculation with CYANA
Güntert, P., & Buchner, L. (2015) · Journal of Biomolecular NMR · doi:10.1007/s10858-015-9924-9 · Cited by 403 · see citing works
High‐accuracy protein structure prediction in <scp>CASP14</scp>
Pereira, J., Simpkin, A. J., Hartmann, M. D., Rigden, D. J., Keegan, R. M., & Lupas, A. N. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26171 · Cited by 314 · see citing works
Shen, Y., & Bax, A. (2014) · Methods in molecular biology · doi:10.1007/978-1-4939-2239-0_2 · Cited by 270 · see citing works
Huang, Y. J., Powers, R., & Montelione, G. T. (2005) · Journal of the American Chemical Society · doi:10.1021/ja047109h · Cited by 253 · see citing works
Self-citationLarge-scale determination of previously unsolved protein structures using evolutionary information
Ovchinnikov, S., Kinch, L., Park, H., Liao, Y., Pei, J., Kim, D. E., Kamisetty, H., Grishin, N. V., & Baker, D. (2015) · eLife · doi:10.7554/elife.09248 · Cited by 222 · see citing works
Recommendations of the wwPDB NMR Validation Task Force
Montelione, G. T., Nilges, M., Bax, A., Güntert, P., Herrmann, T., Richardson, J. S., Schwieters, C. D., Vranken, W., Vuister, G. W., Wishart, D. S., et al. (2013) · Structure · doi:10.1016/j.str.2013.07.021 · Cited by 162 · see citing works
Self-citationSingle Protein Production in Living Cells Facilitated by an mRNA Interferase
Suzuki, M., Zhang, J., Liu, M., Woychik, N. A., & Inouye, M. (2005) · Molecular Cell · doi:10.1016/j.molcel.2005.03.011 · Cited by 145 · see citing works
Self-citationClore, G. M., & Schwieters, C. D. (2004) · Biochemistry · doi:10.1021/bi049357w · Cited by 136 · see citing works
Huang, Y. J., Tejero, R., Powers, R., & Montelione, G. T. (2005) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.20820 · Cited by 131 · see citing works
Self-citationPDBStat: a universal restraint converter and restraint analysis software package for protein NMR
Tejero, R., Snyder, D. A., Mao, B., Aramini, J. M., & Montelione, G. T. (2013) · Journal of Biomolecular NMR · doi:10.1007/s10858-013-9753-7 · Cited by 68 · see citing works
Self-citationComprehensive Automation for NMR Structure Determination of Proteins
Guerry, P., & Herrmann, T. (2011) · Methods in molecular biology · doi:10.1007/978-1-61779-480-3_22 · Cited by 62 · see citing works
A method for validating the accuracy of NMR protein structures
Fowler, N., Sljoka, A., & Williamson, M. P. (2020) · Nature Communications · doi:10.1038/s41467-020-20177-1 · Cited by 61 · see citing works
Wu, N., Kobayashi, N., Tsuda, K., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-76486-4 · Cited by 58 · see citing works
Self-citationObjective identification of residue ranges for the superposition of protein structures
Kirchner, D. K., & Güntert, P. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-170 · Cited by 56 · see citing works
Zhang, D., Li, H., Lin, X., et al. (2015) · FEMS Microbiology Letters · doi:10.1093/femsle/fnv074 · Cited by 55 · see citing works
Improved protein contact predictions with the MetaPSICOV2 server in CASP12
Buchan, D. W. A., & Jones, D. T. (2017) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25379 · Cited by 53 · see citing works
Tarbouriech, N., Ducournau, C., Hutin, S., Mas, P. J., Man, P., Forest, É., Hart, D. J., Peyrefitte, C. N., Burmeister, W. P., & Iseni, F. (2017) · Nature Communications · doi:10.1038/s41467-017-01542-z · Cited by 51 · see citing works
RPF: a quality assessment tool for protein NMR structures
Huang, Y. J., Rosato, A., Singh, G., & Montelione, G. T. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks373 · Cited by 50 · see citing works
Self-citationIslam, M. M., Khan, M. A., & Kuroda, Y. (2012) · Biochimica et Biophysica Acta (BBA) - Proteins and Proteomics · doi:10.1016/j.bbapap.2012.06.005 · Cited by 48 · see citing works
Self-citationRathnayaka, T., Tawa, M., Nakamura, T., et al. (2011) · Biochimica et Biophysica Acta (BBA) - Proteins and Proteomics · doi:10.1016/j.bbapap.2011.09.001 · Cited by 46 · see citing works
Quality assessment of protein NMR structures
Rosato, A., Tejero, R., & Montelione, G. T. (2013) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2013.08.005 · Cited by 35 · see citing works
Self-citationRosato, A., Vranken, W., Fogh, R. H., Ragan, T. J., Tejero, R., Pederson, K., Lee, H., Prestegard, J. H., Yee, A. A., Wu, B., et al. (2015) · Journal of Biomolecular NMR · doi:10.1007/s10858-015-9953-4 · Cited by 29 · see citing works
Self-citationSnyder, D. A., Grullon, J., Huang, Y. J., Tejero, R., & Montelione, G. T. (2013) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.24490 · Cited by 21 · see citing works
Self-citationProtein structure prediction assisted with sparse NMR data in CASP13
Sala, D., Huang, Y. J., Cole, C. A., Snyder, D. A., Liu, G., Ishida, Y., Swapna, G. V. T., Brock, K. P., Sander, C., Fidelis, K. A., et al. (2019) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25837 · Cited by 21 · see citing works
Self-citationStructure prediction using sparse simulated NOE restraints with Rosetta in CASP 11
Ovchinnikov, S., Park, H., Kim, D. E., et al. (2016) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25006 · Cited by 20 · see citing works
Bersch, B., Tarbouriech, N., Burmeister, W. P., et al. (2021) · Journal of Molecular Biology · doi:10.1016/j.jmb.2021.167009 · Cited by 17 · see citing works
Self-citationSchneider, W. M., Inouye, M., Montelione, G. T., et al. (2009) · Journal of Structural and Functional Genomics · doi:10.1007/s10969-009-9067-x · Cited by 16 · see citing works
Self-citationZhang, N., Chang, Y., Tseng, R., Ovchinnikov, S., Schwarz, R., & LiWang, A. (2020) · Protein Science · doi:10.1002/pro.3952 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [2]Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, CA: Curran Associates Inc.; 2017; pp. 6000–6010.
- [42]Bioluminescence SO. Chemical Principles and Methods. 4th ed. World Scientific; 2012.
Cited by
Sorted by citations
Predicting multiple conformations via sequence clustering and AlphaFold2
Wayment-Steele, H. K., Ojoawo, A., Otten, R., Apitz, J. M., Pitsawong, W., Hömberger, M., Ovchinnikov, S., Colwell, L., & Kern, D. (2024) · Nature · doi:10.1038/s41586-023-06832-9 · Cited by 545 · see citing works
Critical assessment of methods of protein structure prediction (CASP)—Round <scp>XIV</scp>
Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K., & Moult, J. (2021) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26237 · Cited by 503 · see citing works
Self-citationThe accuracy of protein structures in solution determined by AlphaFold and NMR
Fowler, N. J., & Williamson, M. P. (2022) · Structure · doi:10.1016/j.str.2022.04.005 · Cited by 104 · see citing works
AlphaFold 2 and NMR Spectroscopy: Partners to Understand Protein Structure, Dynamics and Function
Laurents, D. V. (2022) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2022.906437 · Cited by 95 · see citing works
AlphaFold Models of Small Proteins Rival the Accuracy of Solution NMR Structures
Tejero, R., Huang, Y. J., Ramelot, T. A., & Montelione, G. T. (2022) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2022.877000 · Cited by 56 · see citing works
Self-citationRepresenting structures of the multiple conformational states of proteins
Ramelot, T. A., Tejero, R., & Montelione, G. T. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2023.102703 · Cited by 26 · see citing works
Self-citationVaradi, M., Bordin, N., Orengo, C., & Velankar, S. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2023.102543 · Cited by 29 · see citing works
- Authors
- Huang, Y. J., Zhang, N., Bersch, B., Fidelis, K., Inouye, M., Ishida, Y., Kryshtafovych, A., Kobayashi, N., Kuroda, Y., Liu, G., LiWang, A., Swapna, G. V. T., Wu, N., Yamazaki, T., & Montelione, G. T.
- Affiliations
- Rensselaer Polytechnic Institute (US)
- University of California, Merced (US)
- Centre National de la Recherche Scientifique (FR)
- University of California, Davis (US)
- Rutgers, The State University of New Jersey (US)
- RIKEN (JP)
- Tokyo University of Agriculture and Technology (JP)
- Nexomics Biosciences, Inc. Rocky Hill New Jersey USA
- Zhengzhou University of Light Industry (CN)
- Corresponding author
- Montelione, Gaetano T.
- ORCID
- Venue
- Proteins: Structure, Function, and Bioinformatics
- Publisher
- Wiley
- Published
- 2021-10-19
- Volume
- 89
- Issue
- 12
- Pages
- 1959-1976
- ISSN
- 0887-3585 (ISSN-L)
- 1097-0134
- PMC
- PMC8616817
- OpenAlex
- W3199246256
- Cited by
- 49
- References
- 47
- Citations by year
- 2026: 2 · 2025: 9 · 2024: 10 · 2023: 12 · 2022: 13 · 2021: 2 · 2012: 1
- FWCI
- 2.89
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proteins Structure Function and Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Air Force Office of Scientific Research (FA9550‐17‐1‐0447)
- National Institutes of Health (R01‐GM100482, R01‐GM120574, R35‐GM141818)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Physical Sciences › Materials Science › Materials Chemistry › Enzyme Structure and Function
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Receptor Mechanisms and Signaling
- Keywords
- CASP14, protein structure prediction, solution NMR, AlphaFold2, integral membrane proteins, porin, beta-barrel proteins, protein internal dynamics, NMR data analysis, GDT_TS
- MeSH terms
- Machine Learning, Membrane Proteins, Models, Molecular, Magnetic Resonance Spectroscopy, Protein Conformation, Software, Protein Folding, Computational Biology, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
