Comparative studies of AlphaFold, RoseTTAFold and Modeller: a case study involving the use of G-protein-coupled receptors
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Neural network (NN)-based protein modeling methods have improved significantly in recent years. Although the overall accuracy of the two non-homology-based modeling methods, AlphaFold and RoseTTAFold, is outstanding, their performance for specific protein families has remained unexamined. G-protein-coupled receptor (GPCR) proteins are particularly interesting since they are involved in numerous pathways. This work directly compares the performance of these novel deep learning-based protein modeling methods for GPCRs with the most widely used template-based software—Modeller. We collected the experimentally determined structures of 73 GPCRs from the Protein Data Bank. The official AlphaFold repository and RoseTTAFold web service were used with default settings to predict five structures of each protein sequence. The predicted models were then aligned with the experimentally solved structures and evaluated by the root-mean-square deviation (RMSD) metric. If only looking at each program’s top-scored structure, Modeller had the smallest average modeling RMSD of 2.17 Å, which is better than AlphaFold’s 5.53 Å and RoseTTAFold’s 6.28 Å, probably since Modeller already included many known structures as templates. However, the NN-based methods (AlphaFold and RoseTTAFold) outperformed Modeller in 21 and 15 out of the 73 cases with the top-scored model, respectively, where no good templates were available for Modeller. The larger RMSD values generated by the NN-based methods were primarily due to the differences in loop prediction compared to the crystal structures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
The Phyre2 web portal for protein modeling, prediction and analysis
Kelley, L. A., Mezulis, S., Yates, C. M., Wass, M. N., & Sternberg, M. J. E. (2015) · Nature Protocols · doi:10.1038/nprot.2015.053 · Cited by 9,652 · cited 1×
“Software packages for TBM, such as Modeller [2] and Phyre2 [3], predict accurate structures when good templates are available.”
MethodAccurate prediction of protein structures and interactions using a 3-track network
Baek, M., DiMaio, F., Anishchenko, I., Dauparas, J., Ovchinnikov, S., Lee, G. R., Wang, J., Cong, Q., Kinch, L. N., Schaeffer, R. D., et al. (2021) · doi:10.1101/2021.06.14.448402 · Cited by 5,549
Protein complex prediction with AlphaFold-Multimer
Evans, R., O’Neill, M., Pritzel, A., Antropova, N., Senior, A., Green, T., Žídek, A., Bates, R., Blackwell, S., Yim, J., et al. (2021) · doi:10.1101/2021.10.04.463034 · Cited by 3,808 · cited 1×
“A new version of AlphaFold [27] is being developed as this paper is being prepared, improving modeling accuracy for multimeric proteins.”
BackgroundTM-align: a protein structure alignment algorithm based on the TM-score
Zhang, Y. (2005) · Nucleic Acids Research · doi:10.1093/nar/gki524 · Cited by 3,774
Improved protein structure prediction using potentials from deep learning
Senior, A. W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., Žídek, A., Nelson, A. W. R., Bridgland, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1923-7 · Cited by 3,378
Scoring function for automated assessment of protein structure template quality
Zhang, Y., & Skolnick, J. (2004) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.20264 · Cited by 2,392
Mariani, V., Biasini, M., Barbato, A., & Schwede, T. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt473 · Cited by 1,138
(2009) · doi:10.1007/978-3-540-29678-2 · Cited by 900
Muscarinic acetylcholine receptors: novel opportunities for drug development
Kruse, A. C., Kobilka, B. K., Gautam, D., Sexton, P. M., Christopoulos, A., & Wess, J. (2014) · Nature Reviews Drug Discovery · doi:10.1038/nrd4295 · Cited by 437
Improved protein structure refinement guided by deep learning based accuracy estimation
Hiranuma, N., Park, H., Baek, M., Anishchenko, I., Dauparas, J., & Baker, D. (2021) · Nature Communications · doi:10.1038/s41467-021-21511-x · Cited by 243 · cited 1×
“active/inactive), an additional method [26] might still be required to select the best structures from those generated for both active and inactive states.”
Background
Research works printed but not linked: 17
- Callaway, 2015, The revolution will not be crystallized: a new method sweeps through structural biology, Nature, 525, 172, 10.1038/525172a · DOI · Paper page
- Webb, 2016, Comparative protein structure modeling using MODELLER, Curr Protoc Bioinformatics, 54, 5.6.1-5.6.37, 10.1002/cpbi.3 · DOI · Paper page
- Wu, 2021, Deep template-based protein structure prediction, PLoS Comput Biol, 17, e1008954, 10.1371/journal.pcbi.1008954 · DOI · Paper page
- Chien, 2010, Structure of the human dopamine D3 receptor in complex with a D2/D3 selective antagonist, Science, 330, 1091, 10.1126/science.1197410 · DOI · Paper page
- Doerr, 2010, NMR and the elusive GPCR, Nat Methods, 7, 581, 10.1038/nmeth0810-580b · doi:10.1038/nmeth0810-580b
- Yang, 2021, G protein-coupled receptors: structure-and function-based drug discovery, Signal Transduct Target Ther, 6, 1, 10.1038/s41392-020-00435-w · DOI · Paper page
- Sokoloff, 2006, The dopamine D3 receptor: a therapeutic target for the treatment of neuropsychiatric disorders, CNS Neurol Disord-Drug Targets (Formerly Currt Drug Targets-CNS Neurol Disord), 5, 25
- Maeda, 2020, Structure and selectivity engineering of the M(1) muscarinic receptor toxin complex, Science, 369, 161, 10.1126/science.aax2517 · DOI
- Thal, 2016, Crystal structures of the M1 and M4 muscarinic acetylcholine receptors, Nature, 531, 335, 10.1038/nature17188 · DOI · Paper page
- Weinert, 2017, Serial millisecond crystallography for routine room-temperature structure determination at synchrotrons, Nat Commun, 8, 1, 10.1038/s41467-017-00630-4 · DOI · Paper page
- Lera, 2014, Adenosine A2A receptor as a drug discovery target, J Med Chem, 57, 3623, 10.1021/jm4011669 · DOI · Paper page
- Sun, 2017, Crystal structure of the adenosine A2A receptor bound to an antagonist reveals a potential allosteric pocket, Proc Natl Acad Sci, 114, 2066, 10.1073/pnas.1621423114 · DOI
- Zhou, 2017, Identification of phosphorylation codes for arrestin recruitment by G protein-coupled receptors, Cell, 170, 457, 10.1016/j.cell.2017.07.002 · DOI · Paper page
- Lenahan, 2020, A potential biomarker for neurodegenerative diseases, Front Neurosci, 15, 326, 10.3389/fnins.2020.00326 · DOI
- Zemla, 1999, Processing and analysis of CASP3 protein structure predictions, Proteins Struct Funct Bioinform, 37, 22, 10.1002/(SICI)1097-0134(1999)37:3+<22::AID-PROT5>3.0.CO;2-W · DOI · Paper page
- Olechnovič, 2019, Comparative analysis of methods for evaluation of protein models against native structures, Bioinformatics, 35, 937, 10.1093/bioinformatics/bty760 · DOI · Paper page
- Burley, 2021, Predicting proteome-scale protein structure with artificial intelligence, N Engl J Med, 385, 2191, 10.1056/NEJMcibr2113027 · DOI · Paper page
Cited by
Han, R., Yoon, H., Kim, G., Lee, H., & Lee, Y. (2023) · Pharmaceuticals · doi:10.3390/ph16091259 · Cited by 176
The Epigenetic Dimension of Protein Structure Is an Intrinsic Weakness of the AlphaFold Program
Azzaz, F., Yahi, N., Chahinian, H., & Fantini, J. (2022) · Biomolecules · doi:10.3390/biom12101527 · Cited by 53
“In a recent comparative study focused on G-protein-coupled receptors (GPCR), Lee et al. [49] concluded that the popular template-based method Modeler is superior to both AlphaFold and Robetta (RosseTTAfold) when good templates are available.”
MethodIntegrating Computational Design and Experimental Approaches for Next-Generation Biologics
Son, A., Park, J., Kim, W., Lee, W., Yoon, Y., Ji, J., & Kim, H. (2024) · Biomolecules · doi:10.3390/biom14091073 · Cited by 29
Modeling the Orthosteric Binding Site of the G Protein-Coupled Odorant Receptor OR5K1
Nicoli, A., Haag, F., Marcinek, P., He, R., Kreißl, J., Stein, J., Marchetto, A., Dunkel, A., Hofmann, T., Krautwurst, D., et al. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c00752 · Cited by 28
How accurately can one predict drug binding modes using AlphaFold models?
Karelina, M., Noh, J. J., & Dror, R. O. (2023) · doi:10.7554/elife.89386.1 · Cited by 16
Metadata
- Authors
- Lee, C., Su, B.-H., & Tseng, Y. J.
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-08-09
- Volume
- 23
- Issue
- 5
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- OpenAlex
- W4292458166
- Cited by
- 125
- References
- 11
- FWCI
- 6.97
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Funders
- Taiwan Ministry of Science and Technology (MOST 109-2627-M-002-003-, MOST 110-2320-B-002-038-, MOST 111-2119-M-033-001-)
- Taiwan Food and Drug Administration (MOHW110-FDA-D-114-000611, MOHW111-FDA-D-114-000611)
- National Taiwan University (NTU-CC-110L890803, NTU-110L8809, NTU-CC-111L890203, NTU-111L8809)
- Toxic and Chemical Substances Bureau, Environmental Protection Administration, Executive Yuan (110A022)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.