Nonconjugate Bayesian Estimation of Covariance Matrices and its Use in Hierarchical Models
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
The problem of estimating a covariance matrix in small samples has been considered by several authors following early work by Stein. This problem can be especially important in hierarchical models where the standard errors of fixed and random effects depend on estimation of the covariance matrix of the distribution of the random effects. We propose a set of hierarchical priors (HPs) for the covariance matrix that produce posterior shrinkage toward a specified structure—here we examine shrinkage toward diagonality. We then address the computational difficulties raised by incorporating these priors, and nonconjugate priors in general, into hierarchical models. We apply a combination of approximation, Gibbs sampling (possibly with a Metropolis step), and importance reweighting to fit the models, and compare this hybrid approach to alternative Markov Chain Monte Carlo methods. Our investigation involves three alternative HPs. The first works with the spectral decomposition of the covariance matrix and produces both shrinkage of the eigenvalues toward each other and shrinkage of the rotation matrix toward the identity. The second produces shrinkage of the correlations toward 0, and the third uses a conjugate Wishart distribution to shrink toward diagonality. A simulation study shows that the first two HPs can be very effective in reducing small-sample risk, whereas the conjugate Wishart version sometimes performs very poorly. We evaluate the computational algorithm in the context of a normal nonlinear random-effects model and illustrate the methodology with a logistic random-effects model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A well-conditioned estimator for large-dimensional covariance matrices
Ledoit, O., & Wolf, M. (2004) · Journal of Multivariate Analysis · doi:10.1016/s0047-259x(03)00096-4 · Cited by 3,046
Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods
Smith, A. F. M., & Roberts, G. O. (1993) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1993.tb01466.x · Cited by 1,713 · cited 1×
“The usual approach to posterior simulation in Bayesian hierarchical models is to apply Gibbs sampling, possibly using Metropolis steps to generate individual components (Smith and Roberts 1993).”
MethodThe Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models
Hobert, J. P., & Casella, G. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476714 · Cited by 435 · cited 1×
“For example, Jeffrey's prior in the hierarchical logistic model leads to an improper posterior distribution (Hobert and Casella 1996; Natarajan and McCulloch 1995), and the flat prior can be quite informative for small datasets.”
BackgroundInference for Multivariate Normal Hierarchical Models
Everson, P. J., & Morris, C. N. (2000) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00239 · Cited by 107 · cited 1×
“Two other "noninformative" priors include the Berger-Bernardo prior (Berger and Bernardo 1979; Yang and Berger 1994) and the uniform shrinkage prior (Christiansen and Morris 1997; Daniels 1999; Everson and Morris 1997).”
BackgroundGeneralization of Euler Angles to N -Dimensional Orthogonal Matrices
Hoffman, D. K., Raffenetti, R. C., & Ruedenberg, K. (1972) · Journal of Mathematical Physics · doi:10.1063/1.1666011 · Cited by 69
Research works printed but not linked: 25
- Barnard , J. , McCulloch , R. and Meng , X. 1996 . “A Natural Strategy for Modeling Covariance Matrices With Application to Shrinkage,” , Harvard University, Dept. of Statistics . technical report
- Bennet , J. E. , Racine-Poon , A. and Wakefield , J. C. 1995 . “MCMC for Nonlinear Hierarchical Models,” . In Markov Chain Monte Carlo in Practice , Edited by: Gilks , W. R. , Richardson , S. and Spiegelhalter , D. J. 339 – 358 . New York : Chapman and Hall .
- Berger , J. and Bernardo , J. M. 1992 . “On the Development of Reference Priors” (with discussion) ” . In Bayesian Statistics 4 , Edited by: Bernardo , J. M. , Berger , J. O. , Dawid , A. P. and Smith , A. F. M. 35 – 60 . Oxford , , U.K. : Oxford University Press . · DOI · Paper page
- 10.1080/01621459.1997.10474013 · DOI · Paper page
- Daniels , M. J. 1998 . “Computing Posterior Distributions for Covariance Matrices,” . Computing Science and Statistics , 30 : 192 – 196 .
- 10.2307/3316112 · DOI · Paper page
- Cressie , N. 1998 . “A Hierarchical Approach to Covariance Function Estimation in Time Series,” submitted
- 10.1080/01621459.1999.10473816 · DOI
- 10.1214/aos/1176349756 · DOI · Paper page
- 10.1080/10618600.1997.10474729 · DOI
- 10.1201/b14835 · DOI · Paper page
- Goldstein , H. 1962 . Classical Mechanics , Reading , MA : Addison-Wesley .
- 10.1214/aos/1176348244 · doi:10.1214/aos/1176348244
- 10.1080/01621459.1996.10477003 · DOI · Paper page
- Leonard , T. and Hsu , J. S. 1993 . “Bayesian Inference for a Covariance Matrix,” . The Annals of Statistics , 21 : 1 – 25 .
- Lin , S. P. and Perlman , M. D. 1985 . “A Monte Carlo Comparison of Four Estimators for a Covariance Matrix,” . In Multivariate Analysis 6 , Edited by: Krishnaiah , P. R. 411 – 429 . Amsterdam : North-Holland .
- 10.1093/biomet/82.3.639 · DOI · Paper page
- O'Hagan , A. 1994 . Kendall's Advanced Theory of Statistics, Volume 2B: Bayesian Inference , New York : Halsted Press .
- 10.1007/BF00140873 · DOI · Paper page
- 10.1007/978-1-4612-4250-5 · DOI · Paper page
- Stein , C. “Estimation of a Covariance Matrix,” . Rietz Lecture, 39th Annual meeting IMS . Atlanta , Georgia .
- 10.1080/01621459.1996.10476942 · DOI
- 10.1007/978-1-4684-0192-9 · DOI · Paper page
- Thisted , R. A. 1988 . Elements of Statistical Computing , London : Chapman and Hall .
- 10.1214/aos/1176325625 · DOI · Paper page
Cited by
Bayesian Measures of Model Complexity and Fit
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00353 · Cited by 151
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050
“Noninformative or weakly informative conditionally-conjugate priors could be applied to structured models such as described by Barnard, McCulloch, and Meng (2000) and Daniels and Kass (1999, 2001) , expanded using multiplicative parameters as in Liu (2001) to give the models more flexibility.”
Gaussian Predictive Process Models for Large Spatial Data Sets
Banerjee, S., Gelfand, A. E., Finley, A. O., & Sang, H. (2008) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00663.x · Cited by 997
Uncertainty Quantification and Polynomial Chaos Techniques in Computational Fluid Dynamics
Najm, H. N. (2009) · Annual Review of Fluid Mechanics · doi:10.1146/annurev.fluid.010908.165248 · Cited by 768
A Bayesian analysis of the multinomial probit model with fully identified parameters
McCulloch, R. E., Polson, N. G., & Rossi, P. E. (2000) · Journal of Econometrics · doi:10.1016/s0304-4076(00)00034-8 · Cited by 258
Metadata
- Authors
- Daniels, M. J., & Kass, R. E.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1999-12
- Volume
- 94
- Issue
- 448
- Pages
- 1254-1263
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2162193517
- Cited by
- 221
- References
- 5
- FWCI
- 4.49
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Nonconjugate Bayesian Estimation of Covariance Matrices and its Use in Hierarchical Models. https://app.cochat.ai/papers/10.1080/01621459.1999.10473878
- The paper
- Daniels, M. J., & Kass, R. E. (1999). Nonconjugate Bayesian Estimation of Covariance Matrices and its Use in Hierarchical Models. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1999.10473878
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.