The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities
Abstract
In this work, we discuss practical methods for the assessment, comparison, and selection of complex hierarchical Bayesian models. A natural way to assess the goodness of the model is to estimate its future predictive capability by estimating expected utilities. Instead of just making a point estimate, it is important to obtain the distribution of the expected utility estimate because it describes the uncertainty in the estimate. The distributions of the expected utility estimates can also be used to compare models, for example, by computing the probability of one model having a better expected utility than some other model. We propose an approach using cross-validation predictive densities to obtain expected utility estimates and Bayesian bootstrap to obtain samples from their distributions. We also discuss the probabilistic assumptions made and properties of two practical cross-validation methods, importance sampling and k-fold cross-validation. As illustrative examples, we use multilayer perceptron neural networks and gaussian processes with Markov chain Monte Carlo sampling in one toy problem and two challenging real-world problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
The Predictive Sample Reuse Method with Applications
Geisser, S. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479865 · Cited by 2,748 · cited 2×
“We also discuss the relation of the proposed method to prior predictive densities and Bayes factors (section 3.1), and posterior predictive densities (section 3.2).”
MethodInfluentialModel uncertainty, data mining and statistical inference
Chatfield, C. (1995) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983440 · Cited by 1,103 · cited 1×
“This approach has also been used in some form by Gelfand et al. (1992), Gelfand (1996), and Draper (1995, 1996).”
MethodGood, I. J. (1952) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1952.tb00104.x · Cited by 976
Bayesian approach for neural networks—review and case studies
Lampinen, J., & Vehtari, A. (2001) · Neural Networks · doi:10.1016/s0893-6080(00)00098-8 · Cited by 392
Self-citation
Cited by
Bayesian Measures of Model Complexity and Fit
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00353 · Cited by 151
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611
Self-citationWatanabe, S. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1004.2316 · Cited by 2,226
Understanding predictive information criteria for Bayesian models
Gelman, A., Hwang, J., & Vehtari, A. (2014) · Statistics and Computing · doi:10.1007/s11222-013-9416-2 · Cited by 2,104
“A practical estimate of related sampling uncertainty can be obtained by analyzing the variation in the expected log predictive densities ̂ elppdi using parametric or nonparametric approaches (Vehtari and Lampinen, 2002).”
BackgroundMethodInfluentialSelf-citationJoensuu, H., Vehtari, A., Riihimäki, J., Nishida, T., Steigen, S. E., Brabec, P., Plank, L., Nilsson, B., Cirilli, C., Braconi, C., et al. (2012) · The Lancet Oncology · doi:10.1016/s1470-2045(11)70299-6 · Cited by 997
Self-citation
Metadata
- Authors
- Vehtari, A., & Lampinen, J.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2002-10-01
- Volume
- 14
- Issue
- 10
- Pages
- 2439-2468
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2108868276
- Cited by
- 196
- References
- 6
- FWCI
- 3.68
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities. https://app.cochat.ai/papers/10.1162/08997660260293292
- The paper
- Vehtari, A., & Lampinen, J. (2002). Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities. Neural Computation. https://doi.org/10.1162/08997660260293292
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.