Paintings predict the distribution of species, or the challenge of selecting environmental predictors and evaluation statistics

Abstract

Aim: Species distribution modelling, a family of statistical methods that predicts species distributions from a set of occurrences and environmental predictors, is now routinely applied in many macroecological studies. However, the reliability of evaluation metrics usually employed to validate these models remains questioned. Moreover, the emergence of online databases of environmental variables with global coverage, especially climatic, has favoured the use of the same set of standard predictors. Unfortunately, the selection of variables is too rarely based on a careful examination of the species' ecology. In this context, our aim was to highlight the importance of selecting ad hoc variables in species distribution models, and to assess the ability of classical evaluation statistics to identify models with no biological realism. Innovation: First, we reviewed the current practices in the field of species distribution modelling in terms of variable selection and model evaluation. Then, we computed distribution models of 509 European species using pseudo‐predictors derived from paintings or using a real set of climatic and topographic predictors. We calculated model performance based on the area under the receiver operating curve (AUC) and true skill statistics (TSS), partitioning occurrences into training and test data with different levels of spatial independence. Most models computed from pseudo‐predictors were classified as good and sometimes were even better evaluated than models computed using real environmental variables. However, on average they were better discriminated when the partitioning of occurrences allowed testing for model transferability. Main conclusions: These findings confirm the crucial importance of variable selection and the inability of current evaluation metrics to assess the biological significance of distribution models. We recommend that researchers carefully select variables according to the species' ecology and evaluate models only according to their capacity to be transfered in distant areas. Nevertheless, statistics of model evaluations must still be interpreted with great caution.