Markov Chain Monte Carlo Methods for Computing Bayes Factors: A Comparative Review

Abstract

The problem of calculating posterior probabilities for a collection of competing models and associated Bayes factors continues to be a formidable challenge for applied Bayesian statisticians. Current approaches that take advantage of modern Markov chain Monte Carlo computing methods include those that attempt to sample over some form of the joint space created by the model indicators and the parameters for each model, others that sample over the model space alone, and still others that attempt to estimate the marginal likelihood of each model directly (because the collection of these is equivalent to the collection of model probabilities themselves). We review several methods and compare them in the context of three examples: a simple regression example, a more challenging hierarchical longitudinal model, and a binary data latent variable model. We find that the joint model-parameter space search methods perform adequately but can be difficult to program and tune, whereas the marginal likelihood methods often are less troublesome and require less additional coding. Our results suggest that the latter methods may be most appropriate for practitioners working in many standard model choice settings, but the former remain important for comparing models of varying dimension (e.g., multiple changepoint models) or models whose parameters cannot easily be updated in relatively few blocks. We caution, however, that all methods we compared require significant human and computer effort, and this suggests that less formal Bayesian model choice methods may offer a more realistic alternative in many cases.