Assessment of substitution model adequacy using frequentist and Bayesian methods
Open accessAbstract
In order to have confidence in model-based phylogenetic methods, such as maximum likelihood (ML) and Bayesian analyses, one must use an appropriate model of molecular evolution identified using statistically rigorous criteria. Although model selection methods such as the likelihood ratio test and Akaike information criterion are widely used in the phylogenetic literature, model selection methods lack the ability to reject all models if they provide an inadequate fit to the data. There are two methods, however, that assess absolute model adequacy, the frequentist Goldman-Cox (GC) test and Bayesian posterior predictive simulations (PPSs), which are commonly used in conjunction with the multinomial log likelihood test statistic. In this study, we use empirical and simulated data to evaluate the adequacy of common substitution models using both frequentist and Bayesian methods and compare the results with those obtained with model selection methods. In addition, we investigate the relationship between model adequacy and performance in ML and Bayesian analyses in terms of topology, branch lengths, and bipartition support. We show that tests of model adequacy based on the multinomial likelihood often fail to reject simple substitution models, especially when the models incorporate among-site rate variation (ASRV), and normally fail to reject less complex models than those chosen by model selection methods. In addition, we find that PPSs often fail to reject simpler models than the GC test. Use of the simplest substitution models not rejected based on fit normally results in similar but divergent estimates of tree topology and branch lengths. In addition, use of the simplest adequate substitution models can affect estimates of bipartition support, although these differences are often small with the largest differences confined to poorly supported nodes. We also find that alternative assumptions about ASRV can affect tree topology, tree length, and bipartition support. Our results suggest that using the simplest substitution models not rejected based on fit may be a valid alternative to implementing more complex models identified by model selection methods. However, all common substitution models may fail to recover the correct topology and assign appropriate bipartition support if the true tree shape is difficult to estimate regardless of model adequacy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An adequate modeling of the evolutionary process that generated the data is fundamental to the recovery of accurate phylogenies.
“An adequate modeling of the evolutionary process that generated the data is fundamental to the recovery of accurate phylogenies [16-20].”
From Improved Modeling of Compositional Heterogeneity Supports Sponges as Sister to All Other Animals (2017) · cites this paper as [19]
If the best-fitting model does not describe the evolutionary process that generated the data adequately, there can still be a high probability of recovering a tree with relationships representing systematic error.
“Yet, if the best-fitting model does not describe the evolutionary process that generated the data adequately, there can still be a high probability of recovering a tree with relationships representing systematic error [16-20].”
From Improved Modeling of Compositional Heterogeneity Supports Sponges as Sister to All Other Animals (2017) · cites this paper as [19]
Porifera-sister better represents the phylogenetic signal in the data.
“As phylogenetic methods are less likely to recover artifactual topologies when the models of evolution moreadequatelydescribethedata[16-20],weconcludethatPorifera-sister better represents the phylogenetic signal in the data.”
From Improved Modeling of Compositional Heterogeneity Supports Sponges as Sister to All Other Animals (2017) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033 · see citing works
Kimura, M. (1980) · Journal of Molecular Evolution · doi:10.1007/bf01731581 · Cited by 29,778 · see citing works
MrBayes 3: Bayesian phylogenetic inference under mixed models
Ronquist, F., & Huelsenbeck, J. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg180 · Cited by 28,956 · see citing works
MODELTEST: testing the model of DNA substitution.
Posada, D., & Crandall, K. A. (1998) · Bioinformatics · doi:10.1093/bioinformatics/14.9.817 · Cited by 19,820 · see citing works
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 18,384 · see citing works
Evolutionary trees from DNA sequences: A maximum likelihood approach.
Felsenstein, J. (1981) · Journal of Molecular Evolution · doi:10.1007/bf01734359 · Cited by 14,571 · see citing works
PAUP^* : Phylogenetic Analysis Using Parsimony (and other methods) 4.0b8
Swofford, D. L. (2001) · Cited by 12,789 · see citing works
Tamura, K., & Nei, M. (1993) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a040023 · Cited by 11,267 · see citing works
Evolution of Protein Molecules
JUKES, T. H., & CANTOR, C. R. (1969) · Mammalian Protein Metabolism · doi:10.1016/b978-1-4832-3211-9.50009-7 · Cited by 11,032 · see citing works
Dating of the human–ape splitting by a molecular clock of mitochondrial DNA.
Hasegawa, M., Kishino, H., & Yano, T.-A. (1985) · Journal of Molecular Evolution · doi:10.1007/bf02101694 · Cited by 8,687 · see citing works
PAUP * [Phylogenetic Analysis Using Parsimony (and Other Methods)]
Cummings, M. P. (2004) · Dictionary of Bioinformatics and Computational Biology · doi:10.1002/0471650129.dob0522 · Cited by 5,229 · see citing works
Posada, D., & Buckley, T. R. (2004) · Systematic Biology · doi:10.1080/10635150490522304 · Cited by 3,927 · see citing works
Cases in which Parsimony or Compatibility Methods Will be Positively Misleading
Felsenstein, J. (1978) · Systematic Biology · doi:10.1093/sysbio/27.4.401 · Cited by 3,221 · see citing works
(1964) · Elsevier eBooks · doi:10.1016/b978-1-4832-3210-2.50006-6 · Cited by 3,130 · see citing works
Comparison of phylogenetic trees
Robinson, D. F., & Foulds, L. R. (1981) · Mathematical Biosciences · doi:10.1016/0025-5564(81)90043-2 · Cited by 2,990 · see citing works
Bayesian Inference of Phylogeny and Its Impact on Evolutionary Biology
Huelsenbeck, J. P., Ronquist, F., Nielsen, R., & Bollback, J. P. (2001) · Science · doi:10.1126/science.1065889 · Cited by 2,737 · see citing works
MrModeltest 2.2. Program Distributed by the Author
Nylander, J. A. A., Nylander, J., & Nylander, J. A. A. (2004) · Medical Entomology and Zoology · Cited by 2,648 · see citing works
(1986) · Cited by 2,570 · see citing works
Codon-substitution models for heterogeneous selection pressure at amino acid sites
Yang, Z., Nielsen, R., Goldman, N., & Pedersen, A.-M. K. (2000) · Genetics · doi:10.1093/genetics/155.1.431 · Cited by 2,286 · see citing works
Cases in which parsimony or compatibility methods will be positively misleading.
Felsenstein, J. (1978) · Systematic Zoology · doi:10.2307/2412923 · Cited by 2,230 · see citing works
POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES
Gelman, A., Meng, X., & Stern, H. S. (1996) · Cited by 1,982 · see citing works
A codon-based model of nucleotide substitution for protein-coding DNA sequences.
(1994) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a040153 · Cited by 1,960 · see citing works
Rambaut, A., & Grass, N. C. (1997) · Computer applications in the biosciences · doi:10.1093/bioinformatics/13.3.235 · Cited by 1,545 · see citing works
Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician
Rubin, D. (1984) · The Annals of Statistics · doi:10.1214/aos/1176346785 · Cited by 1,503 · see citing works
Estimation of evolutionary distances between homologous nucleotide sequences.
Kimura, M. (1981) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.78.1.454 · Cited by 1,080 · see citing works
Recovering evolutionary trees under a more realistic model of sequence evolution.
Lockhart, P. J., Steel, M., Hendy, M. D., & Penny, D. (1994) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a040136 · Cited by 964 · see citing works
Statistical tests of models of DNA substitution
Goldman, N. (1993) · Journal of Molecular Evolution · doi:10.1007/bf00166252 · Cited by 736 · see citing works
Modeling compositional heterogeneity
Foster, P. G. (2004) · Systematic Biology · doi:10.1080/10635150490445779 · Cited by 474 · see citing works
Swofford, D. L., Waddell, P. J., Huelsenbeck, J. P., Foster, P. G., Lewis, P. O., & Rogers, J. S. (2001) · Systematic Biology · doi:10.1080/10635150117959 · Cited by 421 · see citing works
Performance-Based Selection of Likelihood Models for Phylogeny Estimation
Minin, V. N., Abdo, Z., Joyce, P., & Sullivan, J. M. (2003) · Systematic Biology · doi:10.1080/10635150390235494 · Cited by 416 · see citing works
Self-citationMolecular phylogenetics: state-of-the-art methods for looking into the past
Whelan, S., Lió, P., & Goldman, N. (2001) · Trends in Genetics · doi:10.1016/s0168-9525(01)02272-7 · Cited by 413 · see citing works
Model Selection in Phylogenetics
Sullivan, J., & Joyce, P. (2005) · Annual Review of Ecology, Evolution, and Systematics · doi:10.1146/annurev.ecolsys.36.102003.152633 · Cited by 412 · see citing works
Self-citationModeling the covarion hypothesis of nucleotide substitution
Tuffley, C., & Steel, M. (1998) · Mathematical Biosciences · doi:10.1016/s0025-5564(97)00081-3 · Cited by 368 · see citing works
The importance of proper model assumption in bayesian phylogenetics.
Lemmon, A. R., & Moriarty, E. C. (2004) · Systematic Biology · doi:10.1080/10635150490423520 · Cited by 356 · see citing works
Accommodating Phylogenetic Uncertainty in Evolutionary Studies
Huelsenbeck, J. P., Rannala, B. H., & Masly, J. P. (2000) · Science · doi:10.1126/science.288.5475.2349 · Cited by 352 · see citing works
The Importance of Data Partitioning and the Utility of Bayes Factors in Bayesian Phylogenetics
Brown, J. M., & Lemmon, A. R. (2007) · Systematic Biology · doi:10.1080/10635150701546249 · Cited by 337 · see citing works
Some mathematical questions in biology : DNA sequence analysis
Miura, R. M. (1986) · American Mathematical Society eBooks · Cited by 323 · see citing works
Are Guinea Pigs Rodents? The Importance of Adequate Models in Molecular Phylogenetics
Sullivan, J. M., & Swofford, D. L. (1997) · Journal of Mammalian Evolution · doi:10.1023/a:1027314112438 · Cited by 283 · see citing works
Self-citationStatistical Methods in Molecular Evolution
Whelan, S., & Nielsen, R. (2006) · Systematic Biology · doi:10.1080/10635150600899780 · Cited by 252 · see citing works
Swofford, D. L., Waddell, P. J., Huelsenbeck, J. P., et al. (2001) · Systematic Biology · doi:10.1080/106351501750435086 · Cited by 248 · see citing works
Success of maximum likelihood phylogeny inference in the four-taxon case.
Gaut, B. S., & Lewis, P. O. (1995) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a040183 · Cited by 245 · see citing works
Sullivan, J. M., Markert, J. A., & Kilpatrick, C. W. (1997) · Systematic Biology · doi:10.1093/sysbio/46.3.426 · Cited by 243 · see citing works
Self-citationData Partitions and Complex Models in Bayesian Analysis: The Phylogeny of Gymnophthalmid Lizards
Castoe, T. A., Doan, T. M., & Parkinson, C. L. (2004) · Systematic Biology · doi:10.1080/10635150490445797 · Cited by 231 · see citing works
A Stochastic Model for the Evolution of Autocorrelated DNA Sequences
Schöniger, M., & von Haeseler, A. (1994) · Molecular Phylogenetics and Evolution · doi:10.1006/mpev.1994.1026 · Cited by 227 · see citing works
Bayesian Model Adequacy and Choice in Phylogenetics
Bollback, J. P. (2002) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a004175 · Cited by 215 · see citing works
Testing a Covariotide Model of DNA Substitution
Huelsenbeck, J. P. (2002) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a004128 · Cited by 205 · see citing works
Evolution of the Mitochondrial Cytochrome Oxidase II Gene in Collembola
Frati, F., Simon, C. M., Sullivan, J. M., & Swofford, D. L. (1997) · Journal of Molecular Evolution · doi:10.1007/pl00006131 · Cited by 193 · see citing works
Self-citationSullivan, J. M., & Swofford, D. L. (2001) · Systematic Biology · doi:10.1080/106351501753328848 · Cited by 182 · see citing works
Self-citationAn Empirical Examination of the Utility of Codon-Substitution Models in Phylogeny Reconstruction
Ren, F., Tanaka, H., & Yang, Z. (2005) · Systematic Biology · doi:10.1080/10635150500354688 · Cited by 168 · see citing works
Model use in phylogenetics: nine key questions
Kelchner, S. A., & Thomas, M. A. (2006) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2006.10.004 · Cited by 158 · see citing works
Buckley, T. R., Simon, C., & Chambers, G. K. (2001) · Systematic Biology · doi:10.1080/10635150116786 · Cited by 149 · see citing works
Reeves, J. H. (1992) · Journal of Molecular Evolution · doi:10.1007/bf00160257 · Cited by 147 · see citing works
Does Choice in Model Selection Affect Maximum Likelihood Analysis?
Ripplinger, J., & Sullivan, J. M. (2008) · Systematic Biology · doi:10.1080/10635150801898920 · Cited by 145 · see citing works
Self-citationSullivan, J. M., Abdo, Z., Joyce, P., et al. (2005) · Molecular Biology and Evolution · doi:10.1093/molbev/msi129 · Cited by 121 · see citing works
Self-citationSanderson, M. J., & Kim, J. (2000) · Systematic Biology · doi:10.1080/106351500750049860 · Cited by 116 · see citing works
Carstens, B. C., Degenhardt, J. D., Stevenson, A. L., & Sullivan, J. M. (2004) · Molecular Ecology · doi:10.1111/j.1365-294x.2004.02404.x · Cited by 103 · see citing works
Self-citationRevell, L. J., Harmon, L. J., & Glor, R. E. (2005) · Systematic Biology · doi:10.1080/10635150500354647 · Cited by 96 · see citing works
FACTORS DETERMINING THE ACCURACY OF CLADOGRAM ESTIMATION: EVALUATION USING COMPUTER SIMULATION
Fiala, K. L., & Sokal, R. R. (1985) · Evolution · doi:10.1111/j.1558-5646.1985.tb00398.x · Cited by 94 · see citing works
Abdo, Z., Minin, V. N., Joyce, P., & Sullivan, J. M. (2004) · Molecular Biology and Evolution · doi:10.1093/molbev/msi050 · Cited by 88 · see citing works
Self-citationBuckley, T. R., Simon, C. M., & Chambers, G. K. (2001) · Systematic Biology · doi:10.1080/106351501750107495 · Cited by 86 · see citing works
Schwarz, M. P., Bull, N. J., & Cooper, S. J. (2003) · Systematic Biology · doi:10.1080/10635150390132632 · Cited by 84 · see citing works
Brändli, L., Handley, L., Vogel, P., & Perrin, N. (2005) · Molecular Phylogenetics and Evolution · doi:10.1016/j.ympev.2005.06.019 · Cited by 67 · see citing works
Demboski, J. R., & Sullivan, J. M. (2003) · Molecular Phylogenetics and Evolution · doi:10.1016/s1055-7903(02)00363-9 · Cited by 64 · see citing works
Self-citationPhytophthora pistaciae sp. nov. and P. melonis: the principal causes of pistachio gummosis in Iran
Mirabolfathy, M., Cooke, D. E. L., Duncan, J. M., Williams, N. A., Ershad, D., & Alizadeh, A. (2001) · Mycological Research · doi:10.1016/s0953-7562(08)61987-5 · Cited by 63 · see citing works
Factors Determining the Accuracy of Cladogram Estimation: Evaluation Using Computer Simulation
Fiala, K. L., & Sokal, R. R. (1985) · Evolution · doi:10.2307/2408656 · Cited by 36 · see citing works
Posterior Mapping and Posterior Predictive Distributions
Bollback, J. P. (2005) · Statistics in the health sciences · doi:10.1007/0-387-27733-1_16 · Cited by 26 · see citing works
A tenth crucial question regarding model use in phylogenetics
Gatesy, J. (2007) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2007.08.002 · Cited by 24 · see citing works
Measuring Fit of Sequence Data to Phylogenetic Model: Gain of Power Using Marginal Tests
Waddell, P. J., Ota, R., & Penny, D. (2009) · Journal of Molecular Evolution · doi:10.1007/s00239-009-9268-8 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Improved Modeling of Compositional Heterogeneity Supports Sponges as Sister to All Other Animals
Feuda, R., Dohrmann, M., Pett, W., Philippe, H., Rota-Stabelli, O., Lartillot, N., Wörheide, G., & Pisani, D. (2017) · Current Biology · doi:10.1016/j.cub.2017.11.008 · Cited by 291 · see citing works
Ekman, S., & Blaalid, R. (2011) · Systematic Biology · doi:10.1093/sysbio/syr022 · Cited by 51 · see citing works
- Authors
- Ripplinger, J., & Sullivan, J.
- Affiliations
- University of Idaho (US)
- Corresponding author
- Ripplinger, Jennifer
- ORCID
- Sullivan, Jack
- Venue
- Molecular Biology and Evolution
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-07-08
- Volume
- 27
- Issue
- 12
- Pages
- 2790-2803
- ISSN
- 0737-4038 (ISSN-L)
- 1537-1719
- PMC
- PMC2981515
- OpenAlex
- W2142150428
- Cited by
- 50
- References
- 68
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 1 · 2023: 1 · 2022: 1 · 2021: 1 · 2020: 3 · 2019: 5 · 2018: 8 · 2017: 1 · 2016: 2 · 2015: 6 · 2014: 5 · 2013: 3 · 2012: 4
- FWCI
- 1.56
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Molecular Biology and Evolution (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Center for Research Resources (P20RR016454, P20RR16448, P20RR016448)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Evolution and Genetic Dynamics
- Keywords
- model selection, posterior predictive simulation, phylogenetic tree topology, branch length estimation, maximum likelihood phylogeny, Bayesian phylogenetics, model fit assessment
- MeSH terms
- Bayes Theorem, Computer Simulation, Models, Genetic, Phylogeny, Research Design, Likelihood Functions, Evolution, Molecular
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
