A simulation‐based evaluation of methods for inferring linear barriers to gene flow

Abstract

Different analytical techniques used on the same data set may lead to different conclusions about the existence and strength of genetic structure. Therefore, reliable interpretation of the results from different methods depends on the efficacy and reliability of different statistical methods. In this paper, we evaluated the performance of multiple analytical methods to detect the presence of a linear barrier dividing populations. We were specifically interested in determining if simulation conditions, such as dispersal ability and genetic equilibrium, affect the power of different analytical methods for detecting barriers. We evaluated two boundary detection methods (Monmonier’s algorithm and WOMBLING), two spatial Bayesian clustering methods (TESS and GENELAND), an aspatial clustering approach (STRUCTURE), and two recently developed, non‐Bayesian clustering methods [PSMIX and discriminant analysis of principal components (DAPC)]. We found that clustering methods had higher success rates than boundary detection methods and also detected the barrier more quickly. All methods detected the barrier more quickly when dispersal was long distance in comparison to short‐distance dispersal scenarios. Bayesian clustering methods performed best overall, both in terms of highest success rates and lowest time to barrier detection, with GENELAND showing the highest power. None of the methods suggested a continuous linear barrier when the data were generated under an isolation‐by‐distance (IBD) model. However, the clustering methods had higher potential for leading to incorrect barrier inferences under IBD unless strict criteria for successful barrier detection were implemented. Based on our findings and those of previous simulation studies, we discuss the utility of different methods for detecting linear barriers to gene flow.