Heritability within groups is uninformative about differences among groups: cases from behavioral, evolutionary, and statistical genetics

Abstract

Without the ability to control or randomize environments (or genotypes), it is difficult to determine the degree to which observed phenotypic differences between two groups of individuals are due to genetic vs. environmental differences. However, some have suggested that these concerns may be limited to pathological cases, and methods have appeared that seem to give—directly or indirectly—some support to claims that aggregate heritable variation within groups can be related to heritable variation among groups. We consider three families of approaches: the “between-group heritability” sometimes invoked in behavior genetics, the statistic P ST used in empirical work in evolutionary quantitative genetics, and methods based on variation in ancestry in an admixed population, used in anthropological and statistical genetics. We take up these examples to show mathematically that information on within-group genetic and phenotypic information in the aggregate cannot separate among-group differences into genetic and environmental components, and we provide simulation results that support our claims. We discuss these results in terms of the long-running debate on this topic.