Intrinsic inference difficulties for trait evolution with Ornstein‐Uhlenbeck models

Abstract

Summary: For the study of macroevolution, phenotypic data are analysed across species on a dated phylogeny using phylogenetic comparative methods. In this context, the Ornstein‐Uhlenbeck (OU) process is now being used extensively to model selectively driven trait evolution, whereby a trait is attracted to a selection optimum μ. We report here theoretical properties of the maximum‐likelihood (ML) estimators for these parameters, including their non‐uniqueness and inaccuracy, and show that theoretical expectations indeed apply to real trees. We provide necessary conditions for ML estimators to be well defined and practical implications for model parametrization. We then show how these limitations carry over to difficulties in detecting shifts in selection regimes along a phylogeny. When the phylogenetic placement of these shifts is unknown, we identify a ‘large p ‐ small n ’ problem where traditional model selection criteria fail and favour overly complex scenarios. Instead, we propose a modified criterion that is better adapted to change‐point models. The challenges we identify here are inherent to trait evolution models on phylogenetic trees when observations are limited to present‐day taxa, and require the addition of fossil taxa to be alleviated. We conclude with recommendations for empiricists.