Interpreting machine-learning models in transformed feature space with an application to remote-sensing classification

Abstract

Model-agnostic tools for the post-hoc interpretation of machine-learning models struggle to summarize the joint effects of strongly dependent features in high-dimensional feature spaces, which play an important role in semantic image classification, for example in remote sensing of landcover. This contribution proposes a novel approach that interprets machine-learning models through the lens of feature-space transformations. It can be used to enhance unconditional as well as conditional post-hoc diagnostic tools including partial-dependence plots, accumulated local effects (ALE) plots, permutation feature importance, or Shapley additive explanations (SHAP). While the approach can also be applied to nonlinear transformations, linear ones are particularly appealing, especially principal component analysis (PCA) and a proposed partial orthogonalization technique. Moreover, structured PCA and model diagnostics along user-defined synthetic features offer opportunities for representing domain knowledge. The new approach is implemented in the R package , which can be combined with existing explainable machine-learning packages. A case study on remote-sensing landcover classification with 46 features is used to demonstrate the potential of the proposed approach for model interpretation by domain experts. It is most useful in situations where groups of feature are linearly dependent and PCA can provide meaningful multivariate data summaries.