Condition-Number-Regularized Covariance Estimation

Abstract

Summary: Estimation of high dimensional covariance matrices is known to be a difficult problem, has many applications and is of current interest to the larger statistics community. In many applications including the so-called ‘large p, small n’ setting, the estimate of the covariance matrix is required to be not only invertible but also well conditioned. Although many regularization schemes attempt to do this, none of them address the ill conditioning problem directly. We propose a maximum likelihood approach, with the direct goal of obtaining a well-conditioned estimator. No sparsity assumptions on either the covariance matrix or its inverse are imposed, thus making our procedure more widely applicable. We demonstrate that the proposed regularization scheme is computationally efficient, yields a type of Steinian shrinkage estimator and has a natural Bayesian interpretation. We investigate the theoretical properties of the regularized covariance estimator comprehensively, including its regularization path, and proceed to develop an approach that adaptively determines the level of regularization that is required. Finally, we demonstrate the performance of the regularized estimator in decision theoretic comparisons and in the financial portfolio optimization setting. The approach proposed has desirable properties and can serve as a competitive procedure, especially when the sample size is small and when a well-conditioned estimator is required.