A Critique of the Bayesian Information Criterion for Model Selection

Abstract

The Bayesian information criterion (BIC) has become a popular criterion for model selection in recent years. The BIC is intended to provide a measure of the weight of evidence favoring one model over another, or Bayes factor. It has, however, some important drawbacks that are not widely recognized. First, Bayes factors depend on prior beliefs about the expected distribution of parameter values, and there is no guarantee that the Bayes factor implied by the BIC will be close to one calculated from a prior distribution that an observer would actually regard as appropriate. Second, to obtain the Bayes factors that follow from the BIC, investigators would have to vary their prior distributions depending on the marginal distributions of the variables and the nature of the hypothesis. Such variations seem unwarranted in principle and tend to make the BIC inclined to favor excessively simple models in practice. These points are illustrated by the analysis of several examples, and alternatives to use of the BIC are discussed.