A phylogenetic approach to comparative genomics

The problem of species non-independence still arises when examining evolutionary labile traits that evolve very fast or are the product of very recent events.

“The problem of species non-independence still arises when examining evolutionary labile traits that evolve very fast or are the product of very recent events.”

• cites [44] Revell, L. J. Phylogenetic signal and linear regression on species data. Methods Ecol. Evol. 1, 319–329 (2010). sourcePartly supported

  The Introduction says species are statistically non-independent because of shared history and explains that phylogenetic dependence in regression residuals can remain even when the traits themselves show very low phylogenetic signal. That supports the general caution, but the paper does not specifically establish it for rapidly evolving traits or traits produced by very recent events. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [54] Losos, J. B. Seeing the forest for the trees: the limitations of phylogenies in comparative biology. Am. Nat. 177, 709–727 (2011). sourceDisputed

  In “Many Traits in Many Clades Do Not Exhibit Phylogenetic Effect” (page unavailable), Losos says rapid evolutionary change can leave closely related species phenotypically dissimilar. The same section says that when a trait lacks phylogenetic effect, values in closely related species may be statistically independent—the contrary of the claim that non-independence still arises for such traits. (LLM, conf=0.88)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [55] Blomberg, S. P., Garland, T. & Ives, A. R. Testing for phylogenetic signal in comparative data: behavioral traits are more labile. Evolution 57, 717–745 (2003). sourceNot checked — no full text of the source
• cites [56] Revell, L. J., Harmon, L. J. & Collar, D. C. Phylogenetic signal, evolutionary process, and rate. Syst. Biol. 57, 591–601 (2008). sourcePartly supported

  The section “Phylogenetic Signal and Evolutionary Rate” (page unavailable) reports that, under unconstrained genetic drift, evolutionary rate has no relationship with phylogenetic signal. This supports the point that rapidly evolving traits cannot simply be assumed independent. But the paper does not establish that non-independence persists for traits produced by very recent events; its discussion of changes concentrated near the tree tips says they can reduce phylogenetic signal, and its introductory passage cautions against assuming dependence without estimating it. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

Across all genomes in the newly curated AllTheBacteria database, the 10 most sequenced species make up 77% of the 1.9 million genomes.

“Across all genomes in the newly curated AllTheBacteria database, the 10 most sequenced species make up 77% of the 1.9 million genomes5.”

• cites [5] Hunt, M., Lima, L., Shen, W., Lees, J. & Iqbal, Z. AllTheBacteria — all bacterial genomes assembled, available and searchable. Preprint at bioRxiv https://doi.org/10.1101/2024.03.08.584059 (2024). sourceDisputed

  The section “A community-built atlas of bacterial and archaeal genomes” reports 2,440,377 genomes in the current release, contrary to the claimed total of 1.9 million. It notes a bias toward frequently sequenced organisms but does not give the claimed 77% share for the ten most sequenced species. (LLM, conf=0.84)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

The problem of non-independence can be exacerbated when examining genomes or genes.

“This problem of non-independence can be exacerbated when examining genomes or genes.”

• Also stated in: § [H1] Introduction · plain

  “In genomic datasets, data points can correspond to genomes or even individual genes, meaning the problem of species not being independent can be exacerbated by additional levels of non-independence, including genes within genomes, such as those carried on the same replicon, and genomes of the same or closely related species16,19,21–26.”

• cites [16] Harvey, P. H. & Pagel, M. D. The Comparative Method in Evolutionary Biology (Oxford Univ. Press, 1991). sourceNot checked — no full text of the source
• cites [19] Grafen, A. The phylogenetic regression. Phil. Trans. R. Soc. Lond. B 326, 119–157 (1989). This paper introduced the phylogenetic regression, a statistical framework to incorporate phylogenetic similarity into regression analyses. sourceNot checked — no full text of the source
• cites [21] Hardison, R. C. Comparative genomics. PLoS Biol. 1, e58 (2003). sourceCould not verify

  The Abstract discusses comparing genomes at different phylogenetic distances, and Prospects notes that related species retain features from a common ancestor. Neither passage addresses whether a problem of non-independence is exacerbated when examining genomes or genes; searches for non-independence and independent observations found no passage making that point. No page numbers are available in our copy. (LLM, conf=0.70)

  Experimental — accuracy not yet measured
• cites [22] Dewar, A. E. et al. Plasmids do not consistently stabilize cooperation across bacteria but may promote broad pathogen host-range. Nat. Ecol. Evol. 5, 1624–1636 (2021). This study used phylogenetic comparative methods and evolutionary theory to test the hypothesis that genes for cooperative traits should be overrepresented on plasmids. sourceSupported
• cites [23] Dewar, A. E., Belcher, L. J., Scott, T. W. & West, S. A. Genes for cooperation are not more likely to be carried by plasmids. Proc. R. Soc. B 291, 20232549 (2024). sourceSupported
• cites [24] Dewar, A. E., Hao, C., Belcher, L. J., Ghoul, M. & West, S. A. Bacterial lifestyle shapes pangenomes. Proc. Natl Acad. Sci. USA 121, e2320170121 (2024). This study used phylogeny-based causal inference methods to identify bacterial lifestyle as the key factor shaping variation in pangenome structure across species. sourcePartly supported

  The Abstract says that species and genomes can share characteristics through common descent and therefore cannot always be treated as independent data points. It does not say that non-independence is exacerbated when examining genomes or genes, or make that comparison explicitly. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [25] Bouvier, J. W. & Kelly, S. Response to Tcherkez and Farquhar: rubisco adaptation is more limited by phylogenetic constraint than by catalytic trade-off. J. Plant. Physiol. 287, 154021 (2023). sourcePartly supported

  The Introduction (page unavailable) says shared ancestry makes biological data points non-independent and suggests that cellular or molecular datasets may be affected by phylogenetic signal. It does not say the problem is exacerbated when examining genomes or genes, or compare its severity with other kinds of data. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites [26] Bouvier, J. W. et al. Rubisco adaptation is more limited by phylogenetic constraint than by catalytic trade-off. Mol. Biol. Evol. 38, 2880–2896 (2021). This study showed that molecular evolution of the enzyme rubisco is mostly influenced by phylogenetic history, contrary to previous work proposing a key role of catalytic trade-offs. sourcePartly supported

  The Introduction explains that rubisco genes share ancestry, making measurements from closely related species non-independent. The Results confirm this for carboxylase-related kinetic traits in angiosperms, but the paper does not establish that non-independence is generally exacerbated when examining genomes or genes. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

The problem of non-independence is complicated even further by how unrepresentative most genomic datasets are of natural communities, owing to biases in which species have been sequenced more frequently.

“The problem of non-independence is complicated even further by how unrepresentative most genomic datasets are of natural communities, owing to biases in which species have been sequenced more frequently (Fig. 2)27–29.”

• cites [27] Blackwell, G. A. et al. Exploring bacterial diversity via a curated and searchable snapshot of archived DNA sequences. PLoS Biol. 19, e3001421 (2021). sourcePartly supported

  The Abstract reports a strongly uneven species composition in the ENA/public bacterial genome databases: 20 of 2,336 species account for 90% of the genomes. The Discussion links this skew to sequencing priorities and says the archive does not capture bacterial diversity well. The paper does not establish that most genomic datasets, beyond the bacterial archives it examines, are unrepresentative of natural communities. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [28] Feng, S. et al. Dense sampling of bird diversity increases power of comparative genomics. Nature 587, 252–257 (2020). sourcePartly supported

  The Abstract says sparse taxon sampling captures only a fraction of genomic diversity, supporting the concern that genomic datasets can be unrepresentative. It does not establish that most datasets are unrepresentative of natural communities because some species are sequenced more frequently, or explain how that bias compounds non-independence. No page numbers are available in our copy. (LLM, conf=0.81)

  Experimental — accuracy not yet measured
• cites [29] Upham, N. S. & Landis, M. J. Genomics expands the mammalverse. Science 380, 358–359 (2023). In a collection of papers, phylogenomics was used to analyse the Zoonomia Consortium’s collection of genomes of 240 diverse mammal species. sourcePartly supported

  In “Genomics expands the mammalverse” (page not provided), the paper reports that available mammal genomes disproportionately represent large-bodied and high-latitude species, and warns that sampling biases can limit the generalizability of evolutionary inferences. It does not establish that most genomic datasets are unrepresentative of natural communities or explain how this compounds non-independence. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

The problem of species not being independent can be exacerbated by additional levels of non-independence, including genes within genomes and genomes of the same or closely related species.

“In genomic datasets, data points can correspond to genomes or even individual genes, meaning the problem of species not being independent can be exacerbated by additional levels of non-independence, including genes within genomes, such as those carried on the same replicon, and genomes of the same or closely related species16,19,21–26.”

• cites [16] Harvey, P. H. & Pagel, M. D. The Comparative Method in Evolutionary Biology (Oxford Univ. Press, 1991). sourceNot checked — no full text of the source
• cites [19] Grafen, A. The phylogenetic regression. Phil. Trans. R. Soc. Lond. B 326, 119–157 (1989). This paper introduced the phylogenetic regression, a statistical framework to incorporate phylogenetic similarity into regression analyses. sourceNot checked — no full text of the source
• cites [21] Hardison, R. C. Comparative genomics. PLoS Biol. 1, e58 (2003). sourceCould not verify

  The Abstract discusses comparisons between genomes at different phylogenetic distances, and the Prospects section defines relationships among homologous genes. Neither passage addresses statistical non-independence among species, genes within a genome, or genomes of closely related species. Searches for non-independence and independent phylogenetic comparisons found no passage making that point; page numbers were not available in our copy. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites [22] Dewar, A. E. et al. Plasmids do not consistently stabilize cooperation across bacteria but may promote broad pathogen host-range. Nat. Ecol. Evol. 5, 1624–1636 (2021). This study used phylogenetic comparative methods and evolutionary theory to test the hypothesis that genes for cooperative traits should be overrepresented on plasmids. sourceSupported
• cites [23] Dewar, A. E., Belcher, L. J., Scott, T. W. & West, S. A. Genes for cooperation are not more likely to be carried by plasmids. Proc. R. Soc. B 291, 20232549 (2024). sourcePartly supported

  The Methods (Data and statistical analyses; page unavailable) warn against grouping genes from multiple genomes or species, use genomes as data points, and account for species and phylogenetic similarity. This supports non-independence across genomes of the same or related species, but the paper does not specifically identify genes on the same replicon as an additional source of non-independence. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [24] Dewar, A. E., Hao, C., Belcher, L. J., Ghoul, M. & West, S. A. Bacterial lifestyle shapes pangenomes. Proc. Natl Acad. Sci. USA 121, e2320170121 (2024). This study used phylogeny-based causal inference methods to identify bacterial lifestyle as the key factor shaping variation in pangenome structure across species. sourcePartly supported

  The Abstract (page unavailable) says that species and genomes can share characteristics through common descent and that species cannot be treated as independent data points. This supports non-independence among related species and genomes, but our copy does not address non-independence of genes within a genome or genes carried on the same replicon. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [25] Bouvier, J. W. & Kelly, S. Response to Tcherkez and Farquhar: rubisco adaptation is more limited by phylogenetic constraint than by catalytic trade-off. J. Plant. Physiol. 287, 154021 (2023). sourcePartly supported

  The Introduction and the section headed “Phylogenetic non-independence” say that shared ancestry makes species-level observations non-independent. Our copy does not address the additional dependence claimed for genes within a genome, genes on the same replicon, or multiple genomes from the same species. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [26] Bouvier, J. W. et al. Rubisco adaptation is more limited by phylogenetic constraint than by catalytic trade-off. Mol. Biol. Evol. 38, 2880–2896 (2021). This study showed that molecular evolution of the enzyme rubisco is mostly influenced by phylogenetic history, contrary to previous work proposing a key role of catalytic trade-offs. sourcePartly supported

  The Introduction explains that Rubisco measurements from closely related species are non-independent because of shared ancestry. The paper does not address the claimed additional levels of non-independence among genes within a genome or among multiple genomes of the same species; the Materials and Methods discuss species-level phylogenetic correction instead. Page numbers were not available in our copy. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Even a slight lack of independence can lead to bias in statistical analyses and incorrect conclusions.

“Even a slight lack of independence can lead to bias in statistical analyses and incorrect conclusions13–15.”

• cites [13] Kruskal, W. Miracles and statistics: the casual assumption of independence. Null 83, 929–940 (1988).Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [14] Ives, A. R. & Zhu, J. Statistics for correlated data: phylogenies, space, and time. Ecol. Appl. 16, 20–32 (2006). sourceNot checked — no full text of the source
• cites [15] Whitney, K. D. & Garland, T. Jr. Did genetic drift drive increases in genome complexity? PLoS Genet. 6, e1001080 (2010). This study re-analysed data on genome size and effective population size while controlling for phylogeny and found no evidence of previously identified correlations, which suggests that these were an artefact of phylogenetic non-independence. sourcePartly supported

  The Introduction says that shared phylogenetic history can violate statistical independence and that nonphylogenetic analyses frequently overestimate associations. Its Figure 1 illustrates an incorrect conclusion, but the paper does not establish that even a slight lack of independence is enough to produce bias or an incorrect conclusion. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Comparative genomic analyses usually test for correlations between traits.

“Comparative genomic analyses usually test for correlations between traits.”

• cites [37] Murray, G. G. R. et al. Genome reduction is associated with bacterial pathogenicity across different scales of temporal and ecological divergence. Mol. Biol. Evol. 38, 1570–1579 (2021). This study used the method of phylogenetic independent contrasts to examine how genome size is correlated with pathogenicity across bacteria. sourcePartly supported

  The Results and Discussion section (‘Pathogenicity Is Associated with Genome Reduction at All Divergence Scales’; page unavailable) reports associations between genomic traits and pathogenicity, and the Statistical Analyses section describes tests of those associations. This supports the paper as an example of comparative genomic analysis testing trait associations, but it does not establish that such analyses usually do so. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites [67] de Albuquerque, N. R. M. et al. A new microsporidian parasite, Ordospora pajunii sp. nov. (Ordosporidae), of Daphnia longispina highlights the value of genomic data for delineating species boundaries. J. Eukaryot. Microbiol. 69, e12902 (2022). sourceNot checked — no full text of the source
• cites [75] Grafen, A. & Hails, R. Modern Statistics for the Life Sciences (Oxford Univ. Press, 2002).Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [76] Zoonomia Consortium. A comparative genomics multitool for scientific discovery and conservation. Nature 587, 240–245 (2020). sourcePartly supported

  The Genetic diversity and extinction risk section tests correlations between genomic diversity measures and species-level traits; Next steps also proposes genotype–phenotype correlations. Neither passage says comparative genomic analyses usually test correlations between traits themselves. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [77] Lloyd-Price, J. et al. Strains, functions and dynamics in the expanded Human Microbiome Project. Nature 550, 61–66 (2017). sourcePartly supported

  The Methods section, “Association testing between microbiome features and phenotypic covariates” (page unavailable), describes testing associations between microbial or pathway abundances and subject metadata. This supports the idea that the paper tests associations, but it does not establish that comparative genomic analyses usually test correlations between traits. (LLM, conf=0.77)

  Experimental — accuracy not yet measured
• cites [78] Duarte, C. M. et al. Sequencing effort dictates gene discovery in marine microbial metagenomes. Environ. Microbiol. 22, 4589–4603 (2020). sourceCould not verify

  The Results section ('Gene discovery across marine and host-associated metagenomes'; page unavailable) compares metagenomic gene catalogues to examine sequencing effort and gene discovery. Searches for comparative genomics and trait correlations found no passage in our copy addressing what comparative genomic analyses usually test. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [79] Arikawa, K. & Hosokawa, M. Uncultured prokaryotic genomes in the spotlight: an examination of publicly available data from metagenomics and single-cell genomics. Comput. Struct. Biotechnol. J. 21, 4508–4518 (2023). sourceCould not verify

  The sections on genetic diversity and prokaryotic genes compare genome datasets, but do not describe what comparative genomic analyses usually test. Searches for trait correlations and associations between genomic features found no passage addressing that claim; page numbers were not available in our copy. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites [80] Rhie, A. et al. Towards complete and error-free genome assemblies of all vertebrate species. Nature 592, 737–746 (2021). sourceCould not verify

  The Methods > Curation passage describes comparative genomics used to check assembly structure, while Fig. 2 reports correlations among genome and assembly measures. Neither passage says that comparative genomic analyses usually test correlations between traits; searches for that formulation and for trait or phenotype correlations found no supporting passage in our copy. (LLM, conf=0.74)

  Experimental — accuracy not yet measured
• cites [81] Blommaert, J. Genome size evolution: towards new model systems for old questions. Proc. R. Soc. B 287, 20201441 (2020). sourcePartly supported

  Section 3, “Potential model organisms for genome size evolution” (page unavailable), says that studies of genome-size evolution often use correlations between genome size and other traits or statistics. It does not establish that comparative genomic analyses generally—beyond this research area—usually test correlations between traits. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [82] Garcia-Garcera, M. & Rocha, E. P. C. Community diversity and habitat structure shape the repertoire of extracellular proteins in bacteria. Nat. Commun. 11, 758 (2020). sourcePartly supported

  The Methods (Statistical analyses; page not provided) describe correlation tests, and the Results (High habitat structure favors extracellular proteins; page not provided) report a correlation between protein-gene frequency and habitat structure. This supports the paper as an example of a genomic analysis testing an association, but it does not establish that comparative genomic analyses usually do so. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Analysing genes or genomes as if they were independent data points can be equivalent to having more technical (pseudo) replicates in an experiment, but then treating each as if they were a truly independent sample (biological replicate).

“Analysing genes or genomes as if they were independent data points can be equivalent to having more technical (pseudo) replicates in an experiment, but then treating each as if they were a truly independent sample (biological replicate)22.”

• cites [22] Dewar, A. E. et al. Plasmids do not consistently stabilize cooperation across bacteria but may promote broad pathogen host-range. Nat. Ecol. Evol. 5, 1624–1636 (2021). This study used phylogenetic comparative methods and evolutionary theory to test the hypothesis that genes for cooperative traits should be overrepresented on plasmids. sourceSupported

Considering the fields of ecology and evolutionary biology, the average % of variance explained in an analysis is 3.6%.

“Considering the fields of ecology and evolutionary biology, the average % of variance explained in an analysis is 3.6%123.”

• cites [123] Gupta, A., Kapil, R., Dhakan, D. B. & Sharma, V. K. MP3: a software tool for the prediction of pathogenic proteins in genomic and metagenomic data. PLoS ONE 9, e93907 (2014). sourceCould not verify

  The Results and Discussions sections report prediction accuracy, sensitivity, specificity, and MCC for pathogenic-protein classifiers, not variance explained across ecology and evolutionary biology analyses. Searches for the claimed 3.6% average and for variance explained in those fields found no passage addressing the claim; page numbers are unavailable in our copy. (LLM, conf=0.77)

  Experimental — accuracy not yet measured

Controlling for confounding variables such as phylogeny does not reduce statistical power.

“Controlling for confounding variables such as phylogeny does not reduce statistical power, because such variables should be accounted for before making any estimates of power18,114,115.”

• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source
• cites [114] Whelan, F. J., Rusilowicz, M. & McInerney, J. O. Coinfinder: detecting significant associations and dissociations in pangenomes. Microb. Genom. 6, e000338 (2020). sourceCould not verify

  The Theory and implementation section describes how Coinfinder assesses phylogenetic lineage-dependence, but does not compare statistical power with and without that assessment. The Results section discusses how Coinfinder’s power varies with dataset size and gene diversity, not whether controlling for phylogeny reduces power or should precede power estimates. No page numbers are available in our copy. (LLM, conf=0.75)

  Experimental — accuracy not yet measured
• cites [115] Pond, S. L. K., Frost, S. D. W. & Muse, S. V. HyPhy: hypothesis testing using phylogenies. Bioinformatics 21, 676–679 (2005). sourceCould not verify

  The Introduction (page unavailable) describes HyPhy as a platform for likelihood-based analyses of sequence alignments, and §2.1.1 describes model fitting and parameter-estimation variability. Neither addresses whether controlling for phylogeny affects statistical power or whether confounders should be accounted for before estimating power; searches for those points found no supporting passage. (LLM, conf=0.74)

  Experimental — accuracy not yet measured

There is also a bias towards bacterial genomes relative to eukaryotes.

“More broadly, there is also a bias towards bacterial genomes relative to eukaryotes3.”

• cites [3] O’Leary, N. A. et al. Reference sequence (RefSeq) database at NCBI: current status, taxonomic expansion, and functional annotation. Nucleic Acids Res. 44, D733–D745 (2016). sourceSupported

More recently developed methods allow analyses to go further and test competing causal hypotheses.

“More recently developed methods allow analyses to go further and test competing causal hypotheses42,84.”

• cites [42] Cornwallis, C. K. & Griffin, A. S. A guided tour of phylogenetic comparative methods for studying trait evolution. Annu. Rev. Ecol. Evol. Syst. 55, 181–204 (2024). A comprehensive review of the history of phylogenetic comparative methods and the latest advances. sourceNot checked — no full text of the source
• cites [84] Shaw, L. P., Rocha, E. P. C. & MacLean, R. C. Restriction-modification systems have shaped the evolution and distribution of plasmids across bacteria. Nucleic Acids Res. 51, 6806–6818 (2023). This phylogenetically controlled analysis of a large plasmid dataset showed that the presence of mechanisms that defend bacteria against mobile genetic elements have influenced how plasmids are distributed across bacteria. sourcePartly supported

  The Materials and Methods sections describe a developed pipeline for predicting R-M targets, a sequence-composition null model, and phylogenetically controlled analyses. These methods support the point that analyses can go further, but the paper does not present them as a general advance for testing competing causal hypotheses. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Genomic databases are biased towards taxonomic groups that have been sequenced more frequently.

“This problem is made worse because genomic databases are biased towards taxonomic groups that have been sequenced more frequently27–29 (Fig. 2).”

• cites [27] Blackwell, G. A. et al. Exploring bacterial diversity via a curated and searchable snapshot of archived DNA sequences. PLoS Biol. 19, e3001421 (2021). sourceSupported
• cites [28] Feng, S. et al. Dense sampling of bird diversity increases power of comparative genomics. Nature 587, 252–257 (2020). sourcePartly supported

  The Abstract (page not provided) says sparse taxon sampling captures only a fraction of genomic diversity, and the Methods describe adding 267 newly sequenced bird genomes to 96 publicly available ones. This supports a concern about limited taxonomic representation, but the paper does not establish the broader claim that genomic databases are biased toward taxonomic groups sequenced more frequently. (LLM, conf=0.74)

  Experimental — accuracy not yet measured
• cites [29] Upham, N. S. & Landis, M. J. Genomics expands the mammalverse. Science 380, 358–359 (2023). In a collection of papers, phylogenomics was used to analyse the Zoonomia Consortium’s collection of genomes of 240 diverse mammal species. sourceSupported

Very large datasets will almost always produce significant P-values, even if the estimated effect size is relatively small.

“Hence, in the extreme, very large datasets will almost always produce significant P-values, even if the estimated effect size is relatively small122 (Fig. 4).”

• cites [122] Uyeda, J. C., Zenil-Ferguson, R. & Pennell, M. W. Rethinking phylogenetic comparative methods. Syst. Biol. 67, 1091–1109 (2018). sourceCould not verify

  Case Study I reports significant regressions driven by a single phylogenetic shift, with significance increasing as the size of that shift increases. It does not say that very large datasets almost always yield significant P-values despite small estimated effects. Searches for sample size, P-values, and effect size did not find that relationship in our copy; the supplied passages have no page numbers. (LLM, conf=0.74)

  Experimental — accuracy not yet measured

Testing whether an ancestral change in one trait consistently leads to the same change in the other trait would suggest the co-evolution is due to a causal link.

“To distinguish which scenario is more likely, we could test whether an ancestral change in one trait consistently leads to the same change in the other trait, which would suggest the co-evolution is due to a causal link42,43,51,85–87 (Fig. 3).”

• cites [42] Cornwallis, C. K. & Griffin, A. S. A guided tour of phylogenetic comparative methods for studying trait evolution. Annu. Rev. Ecol. Evol. Syst. 55, 181–204 (2024). A comprehensive review of the history of phylogenetic comparative methods and the latest advances. sourceNot checked — no full text of the source
• cites [43] Revell, L. J. & Harmon, L. J. Phylogenetic Comparative Methods in R (Princeton Univ. Press, 2022). A very useful book providing a detailed guide for analysing data using phylogenetic comparative methods in the R programming language.Not checked yet

  Not checked in the free run.

• cites [51] Pagel, M. & Meade, A. Bayesian analysis of correlated evolution of discrete characters by reversible‐jump Markov chain Monte Carlo. Am. Nat. 167, 808–825 (2006). sourceNot checked — no full text of the source
• cites [85] Hall, R. J. et al. Gene-gene relationships in an Escherichia coli accessory genome are linked to function and mobility. Microb. Genom. 7, 000650 (2021). sourceCould not verify

  The Introduction discusses gene dependencies, and the Discussion reports repeated gene co-occurrence and avoidance in E. coli. Neither passage addresses testing the order of ancestral trait changes or whether a consistent sequence would indicate a causal link. Searches for ancestral changes, temporal order, and causal relationships found no passage making that point. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites [86] Whelan, F. J., Hall, R. J. & McInerney, J. O. Evidence for selection in the abundant accessory gene content of a prokaryote pangenome. Mol. Biol. Evol. 38, 3697–3708 (2021). sourceCould not verify

  The Methods section describes testing gene-pair presence and absence patterns, not whether ancestral changes in one trait are consistently followed by changes in another. The Discussion says this method misses asymmetrical dependencies and calls for further investigation of direct interactions, but does not propose the claimed ancestral-change test or say that its result would suggest a causal link (page unavailable). (LLM, conf=0.75)

  Experimental — accuracy not yet measured
• cites [87] Hu, E.-Z., Lan, X.-R., Liu, Z.-L., Gao, J. & Niu, D.-K. A positive correlation between GC content and growth temperature in prokaryotes. BMC Genom. 23, 110 (2022). sourcePartly supported

  The Results section ('Evolutionary jumps in bacterial GC contents are correlated with Topt changes'; page not provided) reports that changes in growth temperature and GC content tend to occur in the same direction. The Discussion cautions that not all increases coincide and describes the finding as a statistical correlation. The paper supports testing for coordinated trait changes, but does not say that such a result would suggest a causal link. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Although a phylogeny has been used during the collection of data, it is still important to control for any phylogenetic non-independence when analysing those data across genes, genomes and species.

“Although a phylogeny has been used during the collection of such data, it is still important to control for any phylogenetic non-independence when analysing those data across genes, genomes and species93.”

• cites [93] Boyko, J. D. & Beaulieu, J. M. Generalized hidden Markov models for phylogenetic comparative datasets. Methods Ecol. Evol. 12, 468–478 (2021). sourcePartly supported

  The Worked example and methods (§2.3.2; page unavailable) uses a phylogeny to analyse traits across angiosperm species, supporting the general relevance of phylogenetic analysis. But the paper does not say that using a phylogeny during data collection leaves phylogenetic non-independence to be controlled during analysis, or address that point across genes and genomes. Its discussion of correlated traits (§4) concerns a different kind of dependence. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Grouping by an arbitrary taxonomic rank does not remove the problem of non-independence between the ranks.

“Grouping by an arbitrary taxonomic rank does not remove the problem of non-independence between the ranks19,104.”

• cites [19] Grafen, A. The phylogenetic regression. Phil. Trans. R. Soc. Lond. B 326, 119–157 (1989). This paper introduced the phylogenetic regression, a statistical framework to incorporate phylogenetic similarity into regression analyses. sourceNot checked — no full text of the source
• cites [104] Gavriilidou, A. et al. Goldfinder: unraveling networks of gene co-occurrence and avoidance in bacterial pangenomes. Preprint at bioRxiv https://doi.org/10.1101/2024.04.29.591652 (2024). sourceCould not verify

  The available Abstract discusses phylogenetic dependencies in gene co-occurrence analyses, but it does not address grouping by an arbitrary taxonomic rank or whether that grouping resolves non-independence. Searches for those terms and a paraphrase returned no passage addressing the claim. (LLM, conf=0.72)

  Experimental — accuracy not yet measured

A lack of variation in a potential explanatory variable (X), while the response variable (Y) does vary, is an important result, not a failing of the dataset.

“A lack of variation in a potential explanatory variable (X), while the response variable (Y) does vary, is an important result, not a failing of the dataset24.”

• cites [24] Dewar, A. E., Hao, C., Belcher, L. J., Ghoul, M. & West, S. A. Bacterial lifestyle shapes pangenomes. Proc. Natl Acad. Sci. USA 121, e2320170121 (2024). This study used phylogeny-based causal inference methods to identify bacterial lifestyle as the key factor shaping variation in pangenome structure across species. sourceCould not verify

  The Results section, “Variation across Host-Associated Species” (page unavailable), reports substantial variation in pangenome fluidity and finds that varying lifestyle traits explain some of it. I searched for a lack of variation in an explanatory variable and for whether that would be an important result rather than a dataset failing; no passage in our copy addresses that methodological claim. (LLM, conf=0.73)

  Experimental — accuracy not yet measured

The level of significance (the size of a P-value) will depend on the size of the effect and the number of data points, tested against a null hypothesis of exactly zero difference, or exactly zero correlation.

“Usually, the level of significance (the size of a P-value) will depend on the size of the effect and the number of data points, tested against a null hypothesis of exactly zero difference, or exactly zero correlation119–121.”

• cites [119] Maddison, W. P. & FitzJohn, R. G. The unsolved challenge to phylogenetic correlation tests for categorical characters. Syst. Biol. 64, 127–136 (2015). sourceNot checked — no full text of the source
• cites [120] Kyriacou, R. G., Mulhair, P. O. & Holland, P. W. H. GC content across insect genomes: phylogenetic patterns, causes and consequences. J. Mol. Evol. 92, 138–152 (2024). sourcePartly supported

  The Methods section, “Relation Between GC Content and Chromosome Size” (page unavailable), describes a comparison with a model that forces correlation to zero; the Results section reports significant Pearson correlations using P < 0.05 (page unavailable). The paper does not state that P-value size depends on effect size and the number of data points, and its explicit zero-correlation model is compared using Bayes factors, not P-values. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites [121] Boettiger, C., Coop, G. & Ralph, P. Is your phylogeny informative? Measuring the power of comparative methods. Evolution 66, 2240–2251 (2012). sourcePartly supported

  Methods §2.3 defines a p-value by comparing an observed likelihood difference with its distribution under a null model. Results §4.4 and §5 show that the ability to detect selection depends on its strength and on phylogeny size. The paper does not establish the broader claim that p-values generally depend on the number of data points or that the null is always exactly zero difference or zero correlation; its null is a specified comparative model. Page numbers were not available in our copy. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Particularly successful areas, such as the evolution of sex ratios and cooperation, have produced comparative studies that can explain 20-40% of the variation in data across species.

“Particularly successful areas, such as the evolution of sex ratios and cooperation, have produced comparative studies that can explain 20–40% of the variation in data across species124,125.”

• cites [124] Belcher, L. J. et al. SOCfinder: a genomic tool for identifying social genes in bacteria. Microb. Genom. 9, 001171 (2023). sourceCould not verify

  The Introduction discusses comparing cooperative genes across species, and the Results describe an analysis of genomes from 47 bacterial species. Neither passage says that comparative studies of cooperation or sex ratios explain 20–40% of variation across species. Searches for that figure and for sex-ratio comparative studies found no supporting passage in our copy. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites [125] Belcher, L. J., Dewar, A. E., Ghoul, M. & West, S. A. Kin selection for cooperation in natural bacterial populations. Proc. Natl Acad. Sci. USA 119, e2119070119 (2022). sourcePartly supported

  The Conclusions (page unavailable) say that comparative studies across species found higher cooperation where relatedness is higher. The paper does not report that such studies explain 20–40% of variation across species, and it does not substantiate that figure for sex ratios; searches for the percentage, explained variation, and sex-ratio comparative studies found no matching passage. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Similar problems, and a need for phylogenetic methods, arise when using data from other omic methods, such as transcriptomics, metagenomics or metabolomics.

“Similar problems, and a need for phylogenetic methods, arise when using data from other omic methods, such as transcriptomics, metagenomics or metabolomics61–64.”

• cites [61] Symonds, M. R. E. & Blomberg, S. P. in Modern Phylogenetic Comparative Methods and Their Application in Evolutionary Biology: Concepts and Practice (ed. Garamszegi, L. Z.) Ch. 5, 105–130 (Springer, 2014). sourceNot checked — no full text of the source
• cites [62] Wyatt, C. D. R. et al. Social complexity, life-history and lineage influence the molecular basis of castes in vespid wasps. Nat. Commun. 14, 1046 (2023). sourcePartly supported

  The Discussion (page unavailable) says that lineage influences the molecular toolkit identified from gene-expression data and highlights the importance of considering lineage. The Methods describe phylogenetic analyses of the wasp species. This supports the transcriptomics example, but the paper does not address metagenomics or metabolomics or establish a need for phylogenetic methods across those omics. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [63] Raulo, A. et al. Social and environmental transmission spread different sets of gut microbes in wild mice. Nat. Ecol. Evol. 8, 972–985 (2024). sourcePartly supported

  Methods (Statistical analyses; page not provided) says the authors accounted for phylogenetic non-independence among bacterial genera, supporting the need for a phylogenetic approach in this microbiota analysis. Methods (Microbiota profiling; page not provided) describes 16S rRNA sequencing, not transcriptomics, metagenomics or metabolomics; the paper does not establish the claimed need across those methods. (LLM, conf=0.87)

  Experimental — accuracy not yet measured
• cites [64] Ghoul, M., Andersen, S. B. & West, S. A. Sociomics: using omic approaches to understand social evolution. Trends Genet. 33, 408–419 (2017). sourcePartly supported

  The paper identifies interpretive challenges for omic data in “Challenges to the interpretation of omic data” (p. 13), and “Concluding Remarks” (p. 10) asks whether omic signatures show patterns across phylogenies. “Statistical power in comparative studies” (p. 13) discusses phylogenetically independent transitions, but only for comparative genomic analyses; it does not establish a general need for phylogenetic methods when analysing transcriptomic, metagenomic, or metabolomic data. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

At n=10,000, a relationship explaining only 0.027% of the variance would be significant to P<0.05.

“At n=10,000, a relationship explaining only 0.027% of the variance would be significant to P<0.05.”

73% of bacterial species have only one representative genome in the databases.

“By contrast, 73% of bacterial species have only one representative genome in the databases.”

Of 28,504 complete bacterial genomes in both the RefSeq and Genome Taxonomy Databases, 26% (7,376) are from 0.1% of species.

“For example, of 28,504 complete bacterial genomes in both the RefSeq and Genome Taxonomy Databases, 26% (7,376) are from 0.1% of species (Supplementary Table 1).”

In Fig. 4, even for the very small P-value of P=1x10-4, datasets of ~350 data points or more will produce significant results when the R2 value of the effect is less than 5%.

“To consider a specific case, in Fig. 4, even for the very small P-value of P=1x10-4, datasets of ~350 data points or more will produce significant results when the R2 value of the effect is less than 5%.”

The application of phylogeny-based methods to comparative genomics can address the problem of non-independence.

“The application of phylogeny-based methods to comparative genomics can address this problem.”

• Also stated in: § [H1] Control for phylogenetic relationships · plain

  “The problem of species’ non-independence can be solved by taking the evolutionary history of species into account.”

The problem of species' non-independence can be solved by taking the evolutionary history of species into account.

“The problem of species’ non-independence can be solved by taking the evolutionary history of species into account.”

Closely related species tend to be similar because they share genes by common descent.

“Closely related species tend to be similar because they share genes by common descent, which must be accounted for in analyses.”

• Also stated in: § [H1] Introduction · plain

  “Closely related species tend to be similar because they share traits by common descent rather than through independent evolution (Fig. 1b).”

• cites [16] Harvey, P. H. & Pagel, M. D. The Comparative Method in Evolutionary Biology (Oxford Univ. Press, 1991). sourceNot checked — no full text of the source
• cites [17] Harvey, P. H. & Purvis, A. Comparative methods for explaining adaptations. Nature 351, 619–624 (1991). sourceNot checked — no full text of the source
• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source
• cites [19] Grafen, A. The phylogenetic regression. Phil. Trans. R. Soc. Lond. B 326, 119–157 (1989). This paper introduced the phylogenetic regression, a statistical framework to incorporate phylogenetic similarity into regression analyses. sourceNot checked — no full text of the source
• cites [20] Ridley, M. Why not to use species in comparative tests. J. Theor. Biol. 136, 361–364 (1989). sourceNot checked — no full text of the source

The fact that species cannot be considered independent data points was first recognized in the 1970s by evolutionary biologists studying the evolution of life history traits such as sexual dimorphism and home range in primates.

“The fact that species cannot be considered independent data points was first recognized in the 1970s by evolutionary biologists studying the evolution of life history traits such as sexual dimorphism and home range in primates16,18,30–32.”

• cites [16] Harvey, P. H. & Pagel, M. D. The Comparative Method in Evolutionary Biology (Oxford Univ. Press, 1991). sourceNot checked — no full text of the source
• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source
• cites [30] Clutton‐Brock, T. H. & Harvey, P. H. Primate ecology and social organization. J. Zool. 183, 1–39 (1977). sourceNot checked — no full text of the source
• cites [31] Clutton-Brock, T. H. & Harvey, P. H. Comparison and adaptation. Proc. R. Soc. Lond. B 205, 547–565 (1979). sourceNot checked — no full text of the source
• cites [32] Ridley, M. The Explanation of Organic Diversity: The Comparative Method and Adaptations for Mating (Clarendon Press, 1983).Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Species, genomes, and genes cannot be considered independent data points within statistical tests.

“However, species, genomes and genes cannot be considered as independent data points within statistical tests.”

• Also stated in: § [H1] Introduction · plain

  “A problem for biologists wanting to conduct any form of across-species comparative analyses is that species do not represent independent data points16–20 (Fig. 1a).”

• cites [16] Harvey, P. H. & Pagel, M. D. The Comparative Method in Evolutionary Biology (Oxford Univ. Press, 1991). sourceNot checked — no full text of the source
• cites [17] Harvey, P. H. & Purvis, A. Comparative methods for explaining adaptations. Nature 351, 619–624 (1991). sourceNot checked — no full text of the source
• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source
• cites [19] Grafen, A. The phylogenetic regression. Phil. Trans. R. Soc. Lond. B 326, 119–157 (1989). This paper introduced the phylogenetic regression, a statistical framework to incorporate phylogenetic similarity into regression analyses. sourceNot checked — no full text of the source
• cites [20] Ridley, M. Why not to use species in comparative tests. J. Theor. Biol. 136, 361–364 (1989). sourceNot checked — no full text of the source

Prioritizing the ability to produce significant results ahead of controlling for potential confounding variables can lead to incorrect conclusions not actually supported by the data.

“The general point here is that prioritizing the ability to produce significant results ahead of controlling for potential confounding variables can lead to incorrect conclusions not actually supported by the data.”

Controlling for phylogeny can reduce the chance of detecting a significant result that is not real (Type II error or 'false positive').

“Instead, controlling for phylogeny can reduce the chance of detecting a significant result that is not real (Type II error or ‘false positive’)18,42.”

• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source
• cites [42] Cornwallis, C. K. & Griffin, A. S. A guided tour of phylogenetic comparative methods for studying trait evolution. Annu. Rev. Ecol. Evol. Syst. 55, 181–204 (2024). A comprehensive review of the history of phylogenetic comparative methods and the latest advances. sourceNot checked — no full text of the source

Collecting many more data points from the same phylogenetic cluster could increase any bias due to phylogenetic non-independence.

“Collecting many more data points from the same phylogenetic cluster could increase any bias due to phylogenetic non-independence (Fig. 1)16,18,20.”

• cites [16] Harvey, P. H. & Pagel, M. D. The Comparative Method in Evolutionary Biology (Oxford Univ. Press, 1991). sourceNot checked — no full text of the source
• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source
• cites [20] Ridley, M. Why not to use species in comparative tests. J. Theor. Biol. 136, 361–364 (1989). sourceNot checked — no full text of the source

A result can be statistically significant even when it is unlikely to be biologically important.

“A result can be statistically significant even when it is unlikely to be biologically important.”

If all genomes were used as independent data points, this would increase the bias towards species that have been sequenced the most.

“If all genomes were used as independent data points, this would increase the bias towards species that have been sequenced the most.”

If phylogeny is not controlled for, then increasing the number of data points in this way could make obtaining a spurious result more likely.

“If phylogeny is not controlled for, then increasing the number of data points in this way could make obtaining an spurious result more likely.”

Statistical significance does not necessarily reflect biological importance.

“Statistical significance does not necessarily reflect biological importance.”

The problem of non-independence is not solved by alternative methods such as grouping data by a higher taxonomic rank or by downsampling.

“The problem of non-independence is not solved by alternative methods such as grouping data by a higher taxonomic rank (for example, genera or family), or by sampling the same number of genomes from taxonomic groups to minimize oversampling (‘downsampling’)102,103.”

• cites [102] Eastman, J. M., Alfaro, M. E., Joyce, P., Hipp, A. L. & Harmon, L. J. A novel comparative method for identifying shifts in the rate of character evolution on trees. Evolution 65, 3578–3589 (2011). sourceNot checked — no full text of the source
• cites [103] Garamszegi, L. Z. (ed.) Modern Phylogenetic Comparative Methods and Their Application in Evolutionary Biology: Concepts and Practice (Springer, 2014). sourceNot checked — no full text of the source

There is no simple and robust alternative to controlling for phylogeny.

“There is no simple and robust alternative to controlling for phylogeny, and consequently, the best effort possible must be made to take phylogeny into account, to be able to detect real effects and avoid false positives.”

• Also stated in: § [H2] Do we always need to control for phylogeny? · boosted

  ““Phylogenies are fundamental to comparative biology; there is no doing it without taking them into account.”18”

• cites [18] Felsenstein, J. Phylogenies and the comparative method. Am. Nat. 125, 1–15 (1985). In this key paper, the authors argue that species cannot be considered as independent data points and introduce the method of phylogenetic independent contrasts. sourceNot checked — no full text of the source

Phylogenetic comparative analyses provide a method to control for spurious results that could arise due to even unknown variables.

“Phylogenetic comparative analyses provide a method to control for spurious results that could arise due to even unknown variables.”

It is only possible to control for biases in representation of taxa currently present in a dataset.

“It is, however, only possible to control for biases in representation of taxa currently present in a dataset.”

In terms of sampling bias, phylogenetic methods will control for when taxa are overrepresented, so that better sampled taxa will not have a disproportionately large influence on analyses.

“In terms of sampling bias, phylogenetic methods will control for when taxa are overrepresented, so that better sampled taxa will not have a disproportionately large influence on analyses.”

Closely related species share many features of their ecology and life history.

“Closely related species share many features of their ecology and life history.”

• cites [20] Ridley, M. Why not to use species in comparative tests. J. Theor. Biol. 136, 361–364 (1989). sourceNot checked — no full text of the source

Phylogenetic analyses could also make it more likely to detect a real effect, by controlling for a confounding variable.

“Phylogenetic analyses could also make it more likely to detect a real effect, by controlling for a confounding variable.”

There are cases where the lack of a good phylogeny will pose a problem.

“there are cases where the lack of a good phylogeny will pose a problem112”

• cites [112] Pagel, M. D. & Harvey, P. H. Recent developments in the analysis of comparative data. Q. Rev. Biol. 63, 413–440 (1988). sourceNot checked — no full text of the source

If a response variable (Y) shows considerable variation across species, but a possible explanatory variable (X) does not vary appreciably across species, then variation in Y is not explained by variation in X.

“On the other hand, if we were interested in what explains variation in Y, it is clear that variation in Y is not explained by variation in X.”

There is an almost unlimited number of unknown 'third' variables which might influence the trait of interest.

“Consequently, there is an almost unlimited number of unknown ‘third’ variables which might influence the trait of interest20.”

• cites [20] Ridley, M. Why not to use species in comparative tests. J. Theor. Biol. 136, 361–364 (1989). sourceNot checked — no full text of the source

Even if a response variable shows low (or no) phylogenetic signal, an explanatory variable, known or unknown, could still be correlated with phylogeny.

“Even if a response variable shows low (or no) phylogenetic signal, an explanatory variable, known or unknown, could still be correlated with phylogeny.”