The Geometry of Exponential Families

Abstract

There are two important spaces connected with every multivariate exponential family, the natural parameter space and the expectation parameter space. We describe some geometric results relating the two. (In the simplest case, that of a normal translation family, the two spaces coincide and the geometry is the familiar Euclidean one.) Maximum likelihood estimation, within one-parameter curved subfamilies of the multivariate family, has two simple and useful geometric interpretations. The geometry also relates to the Fisherian question: to what extent can the Fisher information be replaced by $-\partial^2/\partial\theta^2\lbrack\log f_\theta(x)\rbrack\mid_{\theta=\hat{\theta}}$ in the variance bound for $\hat{\theta}$, the maximum likelihood estimator?