CoChat
Sign Up
Cited by 164Genomics and Phylogenetic StudiesRNA and protein synthesis mechanismsPlant Diversity and Evolution

Evaluating fast maximum likelihood-based phylogenetic programs using empirical phylogenomic data sets

Open access
Zhou, Xiaofan, Shen, Xing-Xing, Hittinger, Chris Todd, Rokas, Antonis·2018·Molecular Biology and Evolution
DOI 10.1093/molbev/msx302PMC PMC5850867

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

The sizes of the data matrices assembled to resolve branches of the tree of life have increased dramatically, motivating the development of programs for fast, yet accurate, inference. For example, several different fast programs have been developed in the very popular maximum likelihood framework, including RAxML/ExaML, PhyML, IQ-TREE, and FastTree. Although these programs are widely used, a systematic evaluation and comparison of their performance using empirical genome-scale data matrices has so far been lacking. To address this question, we evaluated these four programs on 19 empirical phylogenomic data sets with hundreds to thousands of genes and up to 200 taxa with respect to likelihood maximization, tree topology, and computational speed. For single-gene tree inference, we found that the more exhaustive and slower strategies (ten searches per alignment) outperformed faster strategies (one tree search per alignment) using RAxML, PhyML, or IQ-TREE. Interestingly, single-gene trees inferred by the three programs yielded comparable coalescent-based species tree estimations. For concatenation-based species tree inference, IQ-TREE consistently achieved the best-observed likelihoods for all data sets, and RAxML/ExaML was a close second. In contrast, PhyML often failed to complete concatenation-based analyses, whereas FastTree was the fastest but generated lower likelihood values and more dissimilar tree topologies in both types of analyses. Finally, data matrix properties, such as the number of taxa and the strength of phylogenetic signal, sometimes substantially influenced the programs' relative performance. Our results provide real-world gene and species tree phylogenetic inference benchmarks to inform the design and execution of large-scale phylogenomic data analyses.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper benchmarks four fast maximum-likelihood phylogenetic programs—RAxML/ExaML, PhyML, IQ-TREE, and FastTree—on 19 empirical phylogenomic data sets with hundreds to thousands of genes and up to 200 taxa. IQ-TREE generally matched or exceeded RAxML/ExaML in likelihood and speed, PhyML scaled poorly to supermatrices, and FastTree was fastest but less accurate.

Key findings

  • For single-gene tree inference, the more exhaustive ten-search strategies performed best: IQ-TREE-10 and RAxML-10 found the highest likelihood scores in 80.17% and 75.99% of alignments, respectively, whereas FastTree did so in only 1.67%.
  • For coalescent-based species tree inference, RAxML-, PhyML-, and IQ-TREE-based strategies produced comparable species trees (median nRF distances 0–0.03), while FastTree was more dissimilar (median nRF 0.121).
  • For concatenation-based species tree inference, IQ-TREE achieved the highest likelihood scores for all 17 analyzed supermatrices; RAxML/ExaML was a close second, PhyML often failed to complete analyses, and FastTree was fastest but yielded lower likelihoods and more dissimilar topologies.
  • Speed comparisons showed FastTree was about 47.9× faster than RAxML on DNA alignments and 95.4× faster on protein alignments; IQ-TREE was about 1.6× faster than RAxML on protein and 1.1× faster on DNA, while PhyML was about 1.5× faster on protein but 3.1× slower on DNA.

Methods

  • The study used 19 empirical phylogenomic data sets from 14 published studies, including 9 amino acid and 10 DNA data sets, with up to 200 taxa and up to 14,446 genes.
  • Seven single-gene strategies were compared: basic RAxML, PhyML, IQ-TREE, and FastTree, plus RAxML-10, PhyML-10, and IQ-TREE-10; species trees were inferred with ASTRAL for coalescent-based analyses and from supermatrices for concatenation-based analyses.
  • Performance was assessed by likelihood scores, normalized Robinson–Foulds distances, and runtime; the authors report more than 670,000 tree inferences on about 45,000 single-gene alignments and supermatrices, costing more than 300,000 CPU hours.

Limitations

  • The paper states that it examined only data sets with up to 200 taxa, so the patterns may not hold for larger matrices with thousands or more taxa; it also notes that true evolutionary histories are unknown for empirical data, so “best-observed” ML trees were used as references rather than direct measures of topological accuracy.
  • The paper states that PhyML results were excluded from concatenation-based evaluation because analyses failed or did not finish on multiple data sets, and that FastTree supports only limited substitution models and is deterministic.
  • Evident from the text: the benchmark is tied to specific program versions (RAxML v8.2.0, PhyML v20160530, IQ-TREE v1.4.2, FastTree v2.1.9) and to particular computational platforms, so relative performance may differ for later versions or other hardware.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/molbev/msx302

Cites

59 of 59 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 42 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    There were strong positive correlations between the speeds of strategies over a wide range of runtimes, with Spearman's correlation ≥0.91 for all combinations of data types and strategies (P < 2.2 × 10−16 in all cases).

    “we then plotted the runtimes of all these strategies against that of RAxML (fig. 4; supplementary table S6, Supplementary Material online), and found strong positive correlations between the speeds of strategies over a wide range of runtimes (Spearman’s correlation ≥0.91 for all combinations of data types and strategies, P-values <2.2 × 10−16 in all cases).”
    Quote in the paperNot a substantial claim
  • Findingown data

    The time advantage of FastTree over RAxML was greater for alignments requiring longer runtimes.

    “In addition, the time advantage of FastTree was greater for alignments requiring longer runtimes”
    Quote in the paperNot a substantial claim
  • Findingown data

    There was a strong positive correlation between differences in likelihood scores and topological distances when comparing inferred trees to best-observed trees, with Spearman's correlations of 0.87 for all alignments and above 0.90 for most data sets (P < 2.2 × 10−16 in all cases).

    “Overall, there was a strong positive correlation between the differences in likelihood scores and the topological distances when comparing inferred trees to the best-observed trees (Spearman’s correlations of 0.87 for all alignments and above 0.90 for most data sets, P-values <2.2 × 10−16 in all cases).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.
  • Findingnone given

    Data matrix properties, such as the number of taxa and the strength of phylogenetic signal, sometimes substantially influenced the programs' relative performance.

    “Finally, data matrix properties, such as the number of taxa and the strength of phylogenetic signal, sometimes substantially influenced the programs’ relative performance.”
    Quote in the paperSubstantial claim
  • Findingown data

    The three programs showed much closer performance when multiple tree searches were conducted.

    “Interestingly, the three programs showed much closer performance when multiple trees searches were conducted.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 60
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 15

  • May combine two claims.

    Findingown data

    FastTree was behind the other strategies in tree topology, leading to median nRF distances greater than 0.33 for most data sets.

    “Here again, FastTree was behind the other strategies as it led to median nRF distances greater than 0.33 for most data sets.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.

  • May combine two claims.

    Findingown data

    Among basic strategies, RAxML performed better than IQ-TREE on only four data sets (MisoA2, StruA5, MisoD2a, and MisoD2b), yet neither of them performed well on these data sets.

    “Among basic strategies, RAxML performed better than IQ-TREE on only four (MisoA2, StruA5, MisoD2a, and MisoD2b) data sets, yet neither of them performed well on these data sets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    FastTree was substantially more time-efficient than RAxML on both DNA alignments (∼47.9 times faster) and protein alignments (∼95.4 times faster).

    “Lastly, FastTree was substantially more time-efficient than RAxML on both DNA alignments (∼47.9 times faster) and protein alignments (∼95.4 times faster).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE-10 and RAxML-10 had the highest frequencies of finding the highest likelihood scores (80.17% and 75.99%, respectively) and reported the highest likelihood scores more frequently than the other strategies in all data sets except for JarvD5b.

    “Overall, IQ-TREE-10 and RAxML-10 had the highest frequencies of finding the highest likelihood scores (80.17% and 75.99%, respectively) and reported the highest likelihood scores more frequently than the other strategies in all data sets except for JarvD5b, for which IQ-TREE-10 performed the best but IQ-TREE slightly outperformed RAxML-10, highlighting the benefit of using multiple starting trees.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On most data sets, IQ-TREE-10 found better likelihood scores more frequently than RAxML-10, which in turn outperformed PhyML-10; the same is true for the basic strategies.

    “The overall trend is the same as that observed in figure 2; on most data sets, IQ-TREE-10 found better likelihood scores more frequently than RAxML-10 which, in turn, outperformed PhyML-10; the same is true for the basic strategies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    FastTree generated lower likelihood values and more dissimilar tree topologies than the other programs in both single-gene and concatenation-based analyses.

    “FastTree was the fastest but generated lower likelihood values and more dissimilar tree topologies in both types of analyses.”
    • Also stated in: § Introduction · boosted
      “In contrast, FastTree produced significantly worse single-gene and species trees than the other three programs even when allowed to run multiple times”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The performances of IQ-TREE-10 and RAxML-10 varied substantially among data sets; on some data sets they performed very similarly, while on others RAxML-10 outperformed IQ-TREE-10 by large margins, or vice versa.

    “Importantly, the performances of IQ-TREE-10 and RAxML-10 varied substantially among data sets; whereas the two strategies performed very similarly on several data sets (e.g., NagyA1 and SongD1), in others RAxML-10 outperformed IQ-TREE-10 by large margins (e.g., MisoA2, MisoD2a, and MisoD2b), or vice versa (e.g., JarvD5b).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Both IQ-TREE and RAxML found higher likelihood scores more often than PhyML in all data sets except for JarvD5b in which RAxML had slightly lower frequency.

    “Both IQ-TREE and RAxML found higher likelihood scores more often than PhyML in all data sets except for JarvD5b in which RAxML had slightly lower frequency.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE outperformed PhyML-10 in 11/19 data sets and showed higher frequency than RAxML-10 in the data set JarvD5b.

    “In fact, IQ-TREE not only outperformed PhyML-10 in 11/19 data sets, but also showed higher frequency than RAxML-10 in the data set JarvD5b, as noted earlier.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    PhyML-10, RAxML, and PhyML performed relatively well in tree topology, with median nRF distances less than 0.03, 0.06, and 0.13, respectively, for ten or more data sets.

    “PhyML-10, RAxML, and PhyML also performed relatively well, with median nRF distances less than 0.03, 0.06, and 0.13, respectively, for ten or more data sets.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.

  • May combine two claims.

    Findingown data

    In some cases, IQ-TREE recovered trees with higher likelihood scores than RAxML/ExaML, although both showed the best performance for most data sets in concatenation-based species tree inference.

    “For the concatenation-based species tree inference, we found that, in some cases, IQ-TREE recovered trees with higher likelihood scores than RAxML/ExaML, although both showed the best performance for most data sets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE exhibited comparable or better speed in both coalescent-based and concatenation-based species tree inference compared with RAxML/ExaML.

    “Importantly, IQ-TREE exhibited comparable or better speed in both coalescent-based and concatenation-based species tree inference compared with RAxML/ExaML.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The basic strategy IQ-TREE was the third best strategy with an overall frequency of 54.03%, slightly higher than that of the more comprehensive strategy PhyML-10 (52.35%).

    “Notably, the basic strategy IQ-TREE was the third best strategy with an overall frequency of 54.03%, slightly higher than that of the more comprehensive strategy PhyML-10 (52.35%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    PhyML was ∼1.5 times faster than RAxML on protein alignments, but ∼3.1 times slower on DNA alignments.

    “Interestingly, PhyML was ∼1.5 times faster than RAxML on protein alignments, but ∼3.1 times slower on DNA alignments.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    PhyML-10 performed consistently better than the basic strategies RAxML and PhyML, whose overall frequencies were 35.98% and 24.17%, respectively.

    “On the other hand, PhyML-10 performed consistently better than RAxML and PhyML, two basic strategies whose overall frequencies were fifth and sixth, respectively, and considerably lower (35.98% and 24.17%) than the first to the fourth best (IQTREE-10, RAxML-10, IQTREE, and PhyML-10).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Linear regression analysis suggests that FastTree might run ∼162.0 times faster than RAxML on the largest single protein alignments but only ∼9.6 times faster on the smallest ones.

    “our linear regression analysis suggests that FastTree might run ∼162.0 times faster than RAxML on the largest single protein alignments but only ∼9.6 times faster on the smallest ones.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Compared with RAxML, IQ-TREE found trees with equally good likelihood scores on 32.67% of all alignments and better scores on 43.96% of all alignments; the frequencies changed to 60.44% and 21.38%, respectively, in the comparison between IQ-TREE-10 and RAxML-10.

    “For instance, compared with RAxML, IQ-TREE found trees with equally good likelihood scores on 32.67% of all alignments and better scores on 43.96% of all alignments; the frequencies changed to 60.44% and 21.38%, respectively, in the comparison between IQ-TREE-10 and RAxML-10.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE was faster than RAxML for both protein and DNA data (∼1.6 and ∼1.1 times faster, respectively).

    “On the contrary, IQ-TREE was faster than RAxML for both protein and DNA data (∼1.6 and ∼1.1 times faster, respectively)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For concatenation-based species tree inference, IQ-TREE consistently achieved the best-observed likelihoods for all data sets, and RAxML/ExaML was a close second.

    “For concatenation-based species tree inference, IQ-TREE consistently achieved the best-observed likelihoods for all data sets, and RAxML/ExaML was a close second.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE-10 and RAxML-10 found higher likelihood scores on 40.27% and 41.77%, respectively, of all alignments than PhyML-10, whereas PhyML-10 found better scores on only ∼12% of all alignments in both comparisons.

    “Nonetheless, IQ-TREE-10 and RAxML-10 still showed considerable advantages over PhyML-10; cumulatively, they found higher likelihood scores on 40.27% and 41.77%, respectively, of all alignments than PhyML-10, whereas PhyML-10 found better scores on only ∼12% of all alignment in both comparisons.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The likelihood scores obtained by FastTree were much lower than those of the other six strategies; FastTree produced the highest likelihood scores in only 1.67% of all alignments.

    “In comparison, the likelihood scores obtained by FastTree were much lower than those of the other six strategies; the program produced the highest likelihood scores in only 1.67% of all alignments.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    FastTree was the fastest among the four programs in both single-gene and concatenation-based analyses.

    “FastTree was the fastest but generated lower likelihood values and more dissimilar tree topologies in both types of analyses.”
    • Also stated in: § Performance Test I: Single-Gene Tree Inference · plain
      “However, FastTree also had substantial advantages in computational speed compared with the others (see below).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When only strongly supported splits (PP ≥ 0.9) were considered, species trees inferred by these strategies became even more similar to the best-observed species trees, although FastTree-generated species trees still showed the greatest topological distances.

    “When we only considered the bipartitions or splits that were strongly supported (i.e., had quartet-based posterior probability, or PP, support greater or equal to 0.9 [Sayyari and Mirarab 2016]), the species tree inferred by these strategies became even more similar to the best-observed species trees, although FastTree-generated species trees still showed the greatest topological distances”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sayyari and Mirarab 2016

      Not checked in the free run.

    • Rests on an assumption: The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.

  • May combine two claims.

    Findingown data

    The median PP supports of incongruent splits for the other six strategies were all below 0.60 for protein and 0.71 for DNA data sets.

    “Both of these values were much higher than those of the other six strategies, which were all below 0.60 and 0.71 for protein and DNA data sets, respectively.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.

  • May combine two claims.

    Findingreasoning

    In single-gene analyses, IQ-TREE outperformed RAxML and PhyML in most instances but did not do so on some data sets with the largest numbers of taxa (MisoA2 and MisoD2a/b, 144 taxa; StruA5, 100 taxa; PrumD6, 200 taxa) when a single tree search was performed.

    “The first one is the number of sequences in the data set; in single-gene analyses, whereas IQ-TREE outperformed RAxML and PhyML in most instances, it did not do so on some of the data sets that had the largest numbers of taxa (MisoA2 and MisoD2a/b, 144 taxa; StruA5, 100 taxa; PrumD6, 200 taxa, when single tree search was performed; supplementary fig. S1, Supplementary Material online).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Many differences in likelihood scores between trees inferred by RAxML/ExaML and IQ-TREE were small; three RAxML and five previously published trees were equally good as the corresponding IQ-TREE trees by approximately unbiased tests.

    “In addition, many of the differences in likelihood scores between trees inferred by RAxML/ExaML and IQ-TREE (the best ML trees) were small; three and five of the RAxML and previously published trees, respectively, were found to be equally good as the corresponding IQ-TREE trees as determined by approximately unbiased tests”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Shimodaira 2002 source

      Not checked in the free run.

    • Rests on an assumption: The RAxML/ExaML-published trees are comparable to trees generated in this study for evaluating likelihood and topology.

  • May combine two claims.

    Findingown data

    The species tree estimations of all six strategies using RAxML, PhyML, or IQ-TREE displayed comparably small topological distances to the best-observed species trees (median nRF distances 0-0.03 across data sets), whereas FastTree species trees were considerably more dissimilar (median nRF 0.121).

    “In that regard, the species tree estimations of all six strategies using RAxML, PhyML, or IQ-TREE displayed comparably small topological distances to the best-observed species trees (median nRF distances ranged between 0 and 0.03 across data sets), whereas the species trees inferred by FastTree were considerably more dissimilar (median nRF distances of 0.121) (table 3).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.

  • May combine two claims.

    Findingown data

    IQ-TREE found the highest likelihood scores in all 17 data sets and outperformed both our RAxML and previous RAxML/ExaML-published results on 7 and 8 data sets, respectively.

    “Interestingly, IQ-TREE found the highest likelihood scores in all 17 data sets and outperformed both our RAxML and previous RAxML/ExaML-published results on 7 and 8 data sets, respectively.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The RAxML/ExaML-published trees are comparable to trees generated in this study for evaluating likelihood and topology.

  • May combine two claims.

    Findingown data

    In PhyML-10, the first tree search found the highest likelihood scores for less than 30% of alignments in 10/19 data sets, and the frequencies increased more evenly with increasing numbers of tree searches.

    “In contrast, the first tree search in PhyML-10 found the highest likelihood scores for much fewer alignments (less than 30% in 10/19 data sets), and the frequencies increased more evenly with increasing numbers of tree searches.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The same pattern of ABS values was observed in the comparison between PhyML-10 and IQ-TREE-10, but not between RAxML-10 and PhyML-10.

    “We also observed the same pattern in the comparison between PhyML-10 and IQ-TREE-10 (supplementary fig. S4C, Supplementary Material online), but not between RAxML-10 and PhyML-10 (supplementary fig. S4A, Supplementary Material online).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Average bootstrap support (ABS) of gene trees is a valid measure of the strength of phylogenetic signal.

  • May combine two claims.

    Findingown data

    In single-gene analyses, PhyML was approximately 1.5 times faster on protein alignments but approximately 3.1 times slower on DNA alignments in comparison with RAxML.

    “in single-gene analyses, PhyML was ∼1.5 times faster on protein alignments but ∼3.1 times slower on DNA alignments in comparison with RAxML”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE consistently yielded the highest likelihood scores in all independent replicates except for data set MisoD2a, whereas RAxML replicates were often trapped at suboptimal solutions.

    “Remarkably, IQ-TREE consistently yielded the highest likelihood scores in all independent replicates (except for the analyses of data set MisoD2a), whereas RAxML replicates were often trapped at suboptimal solutions”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    FastTree was significantly and substantially faster than ExaML and IQ-TREE, whereas ExaML and IQ-TREE were on par with each other with respect to speed.

    “Overall, FastTree was significantly and substantially faster than ExaML and IQ-TREE (Wilcoxon signed-rank test, P-values <0.01 for all pairwise comparisons), whereas the last two programs were on par with each other with respect to speed (Wilcoxon signed-rank test, P-value = 0.56).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In concatenation analyses, IQ-TREE required shorter runtimes (approximately 40%-70%) than RAxML on most protein data sets, whereas runtimes were relatively longer (approximately 60%-110%) for DNA data sets.

    “in concatenation analyses, IQ-TREE required shorter runtimes (∼40%–70%) than RAxML on most protein data sets, whereas the runtimes were relatively longer (∼60%–110%) for DNA data sets”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    IQ-TREE was faster on five of the six protein data sets, whereas ExaML was faster on all four DNA supermatrices.

    “Interestingly, IQ-TREE was faster on five of the six protein data sets, whereas ExaML was faster on all four DNA supermatrices.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Some of the differences between RAxML/ExaML and IQ-TREE disappeared and nRF distances became smaller after NNI-based reoptimization of RAxML/ExaML results.

    “Furthermore, some of these differences disappeared (and nRF distances became smaller) after the NNI-based reoptimization of RAxML/ExaML results.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The RAxML/ExaML-published trees are comparable to trees generated in this study for evaluating likelihood and topology.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In IQ-TREE-10, the highest likelihood scores were found in the first tree search for more than 70% of the alignments in 11/19 data sets, and the frequencies quickly approached 100% with additional tree searches.

    “In IQ-TREE-10, the highest likelihood scores were found in the first tree search for more than 70% of the alignments in 11/19 data sets”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Our single-gene tree inference results are consistent with previous smaller-scale studies on the better efficiency of IQ-TREE relative to RAxML and PhyML and the inferior performance of FastTree in likelihood score maximization.

    “Overall, our results at the level of single-gene tree inference are consistent with previous, smaller-scale studies on the better efficiency of IQ-TREE relative to RAxML and PhyML (all using one search per alignment) (Nguyen et al. 2015), and the inferior performance of FastTree in likelihood score maximization when compared with other programs (Guindon et al. 2010; Liu et al. 2011).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Nguyen et al. 2015 source

      Not checked in the free run.

    • cites Guindon et al. 2010 source

      Not checked in the free run.

    • cites Liu et al. 2011 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Prior to this study, the four programs had only been evaluated using simulated data and empirical data sets with wide ranges of taxon numbers but relatively small numbers of genes, which might not well approximate today's state-of-the-art phylogenomic data matrices.

    “So far, these four programs have only been evaluated using simulated data (Guindon et al. 2010; Price et al. 2010; Liu et al. 2011), and empirical data sets with wide ranges of taxon numbers (e.g., up to 237,882 taxa in Price et al. [2010]) but relatively small numbers of genes (from ∼10 [Price et al. 2010; Liu et al. 2011; Chernomor et al. 2016] to ∼200 [Guindon et al. 2010; Money and Whelan 2012”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Guindon et al. 2010 source

      Not checked in the free run.

    • cites Price et al. 2010 source

      Not checked in the free run.

    • cites Liu et al. 2011 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The 19 data sets were assembled and curated in state-of-the-art phylogenomic studies and are of high quality.

    “In addition, these data sets were assembled and curated in state-of-the-art phylogenomic studies and thus are of high quality.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    ML-based methods are computationally more expensive, necessitating the use of heuristic search algorithms for searching tree space.

    “However, ML-based methods are also computationally more expensive, necessitating the use of heuristic search algorithms for searching the enormity of tree space (Chor and Tuller 2005).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Chor and Tuller 2005

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Liu et al. (2011) examined data sets with highly reliable curated phylogenies as references, which are not available in most empirical studies, and much greater numbers of taxa than the ones examined in our study (up to 200).

    “Importantly, Liu et al. (2011) examined data sets with highly reliable curated phylogenies as references, which are not available in most empirical studies, and also much greater numbers of taxa (between 263 and 27,643 in most cases) than the ones examined in our study (up to 200).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Liu et al. 2011 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The evaluation is based on empirical data sets for which the true evolutionary histories are unknown, preventing a direct measurement of topological accuracy.

    “Our evaluation is based on empirical data sets for which the true evolutionary histories are unknown, thus preventing a direct measurement of topological accuracy.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Statistical methods, such as maximum likelihood, produce more reliable results than distance and parsimony methods.

    “It is now generally accepted that statistical methods, such as maximum likelihood (ML) (Felsenstein 1981), produce more reliable results than distance and parsimony methods (Yang and Rannala 2012; Whelan and Morrison 2017).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Felsenstein 1981 source

      Not checked in the free run.

    • cites Yang and Rannala 2012

      Not checked in the free run.

    • cites Whelan and Morrison 2017

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The difference between our finding that RAxML outperformed PhyML and previous observations might be due to the small number of alignments examined in the previous study and the numerous updates of both programs since then.

    “This difference might be due to the small number of alignments examined in the previous study (Guindon et al. 2010) and the numerous updates of both programs since then.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Guindon et al. 2010 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Finding the best tree is an exceptionally difficult task because evaluation of each tree requires a considerable amount of calculations and because the number of candidate strictly bifurcating trees grows very rapidly with the number of sequences.

    “However, finding the best tree is an exceptionally difficult task because evaluation of each tree requires a considerable amount of calculations (Bryant et al. 2005) as well as because the number of candidate strictly bifurcating trees grows very rapidly with the number of sequences (Felsenstein 1978)—for example, there are ∼8 × 1021 possible rooted topologies for a set of 20 taxa.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Bryant et al. 2005 source

      Not checked in the free run.

    • cites Felsenstein 1978

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In prior studies, RAxML and PhyML showed largely similar performance in identifying trees of higher likelihood scores, whereas IQ-TREE exhibited improved efficiency compared with both RAxML and PhyML.

    “In these studies, RAxML and PhyML showed largely similar performance in identifying trees of higher likelihood scores (Guindon et al. 2010; Money and Whelan 2012), whereas IQ-TREE exhibited improved efficiency compared with both RAxML and PhyML (Nguyen et al. 2015; Chernomor et al. 2016).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Guindon et al. 2010 source

      Not checked in the free run.

    • cites Money and Whelan 2012

      Not checked in the free run.

    • cites Nguyen et al. 2015 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    FastTree was found to be much faster than RAxML and PhyML but reported lower likelihood scores for data sets with both small and large numbers of sequences.

    “On the other hand, FastTree was found to be much faster than RAxML and PhyML but reported lower likelihood scores for data sets with both small and large numbers of sequences (Guindon et al. 2010; Price et al. 2010; Liu et al. 2011).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Guindon et al. 2010 source

      Not checked in the free run.

    • cites Price et al. 2010 source

      Not checked in the free run.

    • cites Liu et al. 2011 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    RAxML-10 and PhyML-10 would take about ten times longer than RAxML and PhyML, respectively.

    “suggesting that RAxML-10 and PhyML-10 would take about ten times longer than RAxML and PhyML, respectively”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Independent likelihood score re-calculations were conducted for all trees using IQ-TREE v.1.4.2 and the R package phangorn v2.2.0 to control for potential biases, since RAxML itself is one of the programs being assessed.

    “Independent likelihood score re-calculations were conducted for all trees using IQ-TREE v.1.4.2 and the R package “phangorn” v2.2.0 to control for potential biases since RAxML itself is one of the programs to be assessed.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Trees of the same topology have the same likelihood score.

  • May combine two claims.

    Interpretationown data

    The two types of starting trees are similarly efficient in the analysis of single-gene alignments with moderate sequence numbers, although random starting trees might be slightly more advantageous.

    “Together with their similar run-time performances (fig. 4), these results suggest that the two types of starting trees are similarly efficient in the analysis of single-gene alignments with moderate sequence numbers, although random starting trees might be slightly more advantageous.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    RAxML and ExaML results, even after multiple independent searches, should not be directly taken as the best answers and should be checked for potential improvements.

    “For RAxML and ExaML, our findings indicate that their results, even after multiple independent searches, should not be directly taken as the best answers and instead should be checked for potential improvements.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The RAxML/ExaML-published trees are comparable to trees generated in this study for evaluating likelihood and topology.

  • May combine two claims.

    Premisecitation

    IQ-TREE uses NNI as its topological rearrangement mechanism, whereas RAxML and PhyML are both based on SPR.

    “A potential explanation could be that IQ-TREE uses NNI as its topological rearrangement mechanism (Nguyen et al. 2015), whereas RAxML and PhyML are both based on SPR (Stamatakis 2006; Guindon et al. 2010).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Nguyen et al. 2015 source

      Not checked in the free run.

    • cites Stamatakis 2006 source

      Not checked in the free run.

    • cites Guindon et al. 2010 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The findings of this study are directly relevant for the experimental design and execution of real-world phylogenetic and, particularly, phylogenomic studies.

    “our findings are directly relevant for the experimental design and execution of real-world phylogenetic and, particularly, phylogenomic studies”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    JarvD5b was derived from concatenating single-gene alignments in JarvD5a into a smaller number of longer partitions, resulting in enhanced phylogenetic signal measured by average bootstrap support (ABS) of gene tree.

    “JarvD5b was derived from concatenating single-gene alignments in JarvD5a into a smaller number of longer partitions (Mirarab et al. 2014), resulting in enhanced phylogenetic signal (measured by average bootstrap support, or ABS, of gene tree)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Mirarab et al. 2014 source

      Not checked in the free run.

    • Rests on an assumption: Average bootstrap support (ABS) of gene trees is a valid measure of the strength of phylogenetic signal.

  • May combine two claims.

    Premisecitation

    SPR explores a greater proportion of tree space than NNI and does so in a manner proportional to the sequence number (SPR examines O(n^2) neighbors for each tree instead of O(n) neighbors by NNI).

    “It is well recognized that SPR explores a greater proportion of tree space than NNI (Whelan and Morrison 2017) and that it does so in a manner proportional to the sequence number (SPR examines On2 neighbors for each tree instead of On neighbors by NNI).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Whelan and Morrison 2017

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    IQ-TREE exhibited better performance on data sets with fewer taxa through a combination of NNI rearrangement and stochastic algorithm, but NNI might become a limiting factor on its performance on larger data sets.

    “Therefore, whereas IQ-TREE exhibited better performance on data sets with fewer taxa through a combination of NNI rearrangement and stochastic algorithm, NNI might become a limiting factor on its performance on larger data sets.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The exceptional runtime of FastTree might make it an attractive option for exploratory investigations, but its results should still be interpreted with care.

    “The exceptional runtime of FastTree might make it an attractive option for exploratory investigations, yet the results should still be interpreted with care.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Study design and conductreasoning

    To ensure fairness of the comparison, the likelihood scores of all trees were re-calculated using RAxML v8.2.0 with models set to the best-fit models (or the respective partition schemes for supermatrices).

    “To ensure the fairness of the comparison, the likelihood scores of all trees were re-calculated using RAxML v8.2.0 with models set to the best-fit models”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Trees of the same topology have the same likelihood score.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    IQ-TREE represents the latest development in fast phylogenetic programs and has implemented a novel data structure to facilitate concatenation analysis.

    “This finding might not be entirely surprising because IQ-TREE represents the latest development in fast phylogenetic programs and has implemented a novel data structure to facilitate concatenation analysis (Chernomor et al. 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chernomor et al. 2016 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    PhyML required exceedingly high runtime, memory, or crashed on multiple data sets, so its results were not included in the concatenation-based evaluation.

    “However, as PhyML required exceedingly high runtime, memory, or crashed on multiple data sets, its results are not included in the evaluation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    The algorithm of FastTree is deterministic, so independent analyses of the same alignment always lead to the same result.

    “Moreover, the algorithm of FastTree is deterministic, thus independent analyses of the same alignment will always lead to the same result.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    The benchmarking is based on empirical data sets whose true underlying histories were unknown, which prevents a direct measurement of the topological accuracy of the programs.

    “Our benchmarking is based on empirical data sets whose true underlying histories were unknown, thus preventing a direct measurement of the topological accuracy of programs.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To address the non-comparability of runtimes across computational platforms, 10% of single-gene alignments were randomly selected from each data set and all relevant analyses were redone on the same type of ACCRE computing nodes.

    “we selected 10% of single-gene alignments randomly from each data set and redid all relevant phylogenetic analyses on Vanderbilt University’s ACCRE cluster”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The empirical data sets were used in this study without any filtering on their contents.

    “They were used in this study without any filtering on their contents”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The BFGS optimization method has been reported previously to produce unstable results on nucleotide alignments.

    “The BFGS optimization method was turned off in the analyses of nucleotide alignments since it has been reported previously to produce unstable results (Church et al. 2015).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Church et al. 2015 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Because RAxML is a subset of the searches conducted in RAxML-10, RAxML-10 will always produce equal or better results than RAxML.

    “Therefore, RAxML-10 will always produce equal or better results than RAxML.”
    Quote in the paperSubstantial claim
67 of 67

  • The paper relies on this without support.

    Trees with similar likelihood scores may differ substantially in their topologies, or vice versa.

    “Trees with similar likelihood scores may differ substantially in their topologies, or vice versa.”

  • The authors state this as an assumption.

    Under the ML optimality criterion, the topologies of the trees with the highest likelihood scores are considered the best currently known answer, so the best-observed ML tree can serve as the reference for topological comparisons.

    “The rationale for using the best-observed ML trees as the references in our comparison is that, under the ML optimality criterion (which underlies all the methods examined here), the topologies of the trees with the highest likelihood scores are considered the best (currently known) answer.”

  • The paper relies on this without support.

    The best-observed trees and best-observed species trees are suitable references for evaluating topological accuracy.

    “We measured the normalized Robinson–Foulds, or nRF, distances (Robinson and Foulds 1981) between trees inferred by the seven strategies on each alignment against the corresponding best-observed tree.”

  • The paper relies on this without support.

    The RAxML/ExaML-published trees are comparable to trees generated in this study for evaluating likelihood and topology.

    “we included the reported likelihood scores and topologies—we refer to them as “RAxML/ExaML-published” trees—in our examination of relative performance.”

  • The paper relies on this without support.

    Average bootstrap support (ABS) of gene trees is a valid measure of the strength of phylogenetic signal.

    “The average bootstrap values are taken from previously reported phylogenies for the alignments are used here as a measure of the strength of phylogenetic signal.”

  • The authors state this as an assumption.

    Trees of the same topology have the same likelihood score.

    “Trees of the same topology are presumed to have the same likelihood score.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • More accurate multiple sequence alignment and tree inference methods should give more accurate ortholog inference.

    “More accurate multiple sequence alignment and tree inference methods should give more accurate ortholog inference, and many studies exist comparing the accuracy and runtime characteristics of the available methods [36, 37].”

    From OrthoFinder: phylogenetic orthology inference for comparative genomics (2019) · cites this paper as [37]

    The Results and Discussion (Performance Test I: Single-Gene Tree Inference; page unavailable) compares tree-inference methods, and the Materials and Methods (Assessment of Tree Inferences; page unavailable) describes runtime comparisons. The paper therefore supports the citing sentence’s point that such methods have been benchmarked, but our copy does not link more accurate alignments or trees to more accurate ortholog inference. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • The speed of FastTree2 was accomplished at the cost of producing substantially worse trees than RAxML, PhyML, and IQ-TREE.

    “The speed of FastTree2 was accomplished at the cost of producing substantially worse trees than RAxML, PhyML, and IQ-TREE (Zhou et al. 2018).”

    From IQ-TREE 2: New Models and Efficient Methods for Phylogenetic Inference in the Genomic Era (2020) · cites this paper as Zhou et al. 2018

  • Zhou et al. (2018) independently showed that the tree search algorithm in IQ-TREE exhibits good performance in terms of both computing times and likelihood maximization when compared with other popular ML phylogenetics software such as RAxML and PhyML.

    “Zhou et al. (2018) independently showed that the tree search algorithm in IQ-TREE exhibits good performance in terms of both computing times and likelihood maximization when compared with other popular ML phylogenetics software such as RAxML (Stamatakis 2014) and PhyML (Guindon et al. 2010).”

    From IQ-TREE 2: New Models and Efficient Methods for Phylogenetic Inference in the Genomic Era (2020) · cites this paper as Zhou et al. 2018

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Concatenation results name an excluded data set

    Minor

    The paper says concatenation analyses excluded MisoD2a because it had no corresponding supermatrix, but later describes IQ-TREE concatenation replicates for MisoD2a.

  • Claim that RAxML outperformed PhyML in every data set conflicts with stated exception

    Minor

    The single-gene results identify JarvD5b as a data set where RAxML had a lower frequency of finding the highest likelihood score than PhyML, contrary to the later claim about all data sets.

  • Figure 8 names RAxML where the runtime comparison names ExaML

    Minor

    The runtime-results text identifies ExaML as one of the three programs compared on ten supermatrices, while the Figure 8 caption identifies RAxML instead.

  • JarvD5a gene count differs from Table 2

    Minor

    The text gives JarvD5a as having 14,448 genes, but Table 2 lists 14,446.

4 of 4

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Zhou, X., Shen, X.-X., Hittinger, C. T., & Rokas, A.
Affiliations
  • South China Agricultural University (CN)
  • Vanderbilt University (US)
  • University of Wisconsin–Madison (US)
Corresponding author
Rokas, Antonis
ORCID
Venue
Molecular Biology and Evolution
Publisher
Oxford University Press (OUP)
Published
2017-11-21
Volume
35
Issue
2
Pages
486-503
ISSN
  • 0737-4038 (ISSN-L)
  • 1537-1719
DOI
10.1093/molbev/msx302
PMC
PMC5850867
OpenAlex
W2949367299
Semantic Scholar
89fbcfd6a6979fde375f88453423bb2a5c4186b8
Cited by
164
References
59
Citations by year
2026: 6 · 2025: 10 · 2024: 19 · 2023: 15 · 2022: 20 · 2021: 20 · 2020: 25 · 2019: 33 · 2018: 16
FWCI
8.27
Type
journal article
Language
en
Open access
Licence
Locations
  • Molecular Biology and Evolution (publishedVersion, cc-by-nc)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
National Science Foundation (DEB-1442113, DEB-1442148)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
  • Life Sciences › Agricultural and Biological Sciences › Ecology, Evolution, Behavior and Systematics › Plant Diversity and Evolution
Keywords
maximum likelihood estimation, RAxML, PhyML, IQ-TREE, FastTree, genome-scale phylogenomics, species tree estimation, coalescent, gene tree inference, likelihood maximization, tree topology, computational speed, benchmarking, phylogenetic signal strength
MeSH terms
Phylogeny, Software, Likelihood Functions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review