The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Comparison of Bayesian predictive methods for model selection
Open accessAbstract
The goal of this paper is to compare several widely used Bayesian model selection methods in practical model selection problems, highlight their differences and give recommendations about the preferred approaches. We focus on the variable subset selection for regression and classification and perform several numerical experiments using both simulated and real world data. The results show that the optimization of a utility estimate such as the cross-validation (CV) score is liable to finding overfitted models due to relatively high variance in the utility estimates when the data is scarce. This can also lead to substantial selection induced bias and optimism in the performance evaluation for the selected model. From a predictive viewpoint, best results are obtained by accounting for model uncertainty by forming the full encompassing model, such as the Bayesian model averaging solution over the candidate models. If the encompassing model is too complex, it can be robustly simplified by the projection method, in which the information of the full model is projected onto the submodels. This approach is substantially less prone to overfitting than selection based on CV-score. Overall, the projection method appears to outperform also the maximum a posteriori model and the selection of the most probable variables. The study also demonstrates that the model selection can greatly benefit from using cross-validation outside the searching process both for guiding the model size selection and assessing the predictive performance of the finally selected model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611
Self-citationHoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118 · cited 4×
“Our results agree with what is known about the good performance of the BMA (Hoeting et al., 1999; Raftery and Zheng, 2003).”
MethodResultInfluentialVariable Selection via Gibbs Sampling
George, E. I., & McCulloch, R. E. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476353 · Cited by 2,828
Bias in error estimation when using cross-validation for model selection
Varma, S., & Simon, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-91 · Cited by 2,485 · cited 1×
“However, the topic has been discussed for example by Rencher and Pun (1980), Ambroise and McLachlan (2002), Reunanen (2003), Varma and Simon (2006), and Cawley and Talbot (2010).”
BackgroundBernardo, J. M., & Smith, A. F. M. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316870 · Cited by 2,178
Good, I. J. (1952) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1952.tb00104.x · Cited by 976
Cited by
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611
Self-citationForecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Ben Taieb, S., Bergmeir, C., Bessa, R. J., Bijak, J., Boylan, J. E., et al. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 921
Using Stacking to Average Bayesian Predictive Distributions (with Discussion)
Yao, Y., Vehtari, A., Simpson, D., & Gelman, A. (2018) · Bayesian Analysis · doi:10.1214/17-ba1091 · Cited by 493
Self-citationCross validation for model selection: A review with examples from ecology
Yates, L. A., Aandahl, Z., Richards, S. A., & Brook, B. W. (2023) · Ecological Monographs · doi:10.1002/ecm.1557 · Cited by 459
Model Averaging and Its Use in Economics
Steel, M. F. J. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191385 · Cited by 321
Metadata
- Authors
- Piironen, J., & Vehtari, A.
- Venue
- Statistics and Computing
- Published
- 2016-04-07
- Volume
- 27
- Issue
- 3
- Pages
- 711-735
- ISSN
- 0960-3174 (ISSN-L)
- 1573-1375
- OpenAlex
- W1877880381
- Cited by
- 294
- References
- 7
- FWCI
- 23.70
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Comparison of Bayesian predictive methods for model selection. https://app.cochat.ai/papers/10.1007/s11222-016-9649-y
- The paper
- Piironen, J., & Vehtari, A. (2017). Comparison of Bayesian predictive methods for model selection. Statistics and Computing. https://doi.org/10.1007/s11222-016-9649-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.