Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
This paper studies the multiplicity-correction effect of standard Bayesian variable-selection priors in linear regression. Our first goal is to clarify when, and how, multiplicity correction happens automatically in Bayesian analysis, and to distinguish this correction from the Bayesian Ockham’s-razor effect. Our second goal is to contrast empirical-Bayes and fully Bayesian approaches to variable selection through examples, theoretical results and simulations. Considerable differences between the two approaches are found. In particular, we prove a theorem that characterizes a surprising aymptotic discrepancy between fully Bayes and empirical Bayes. This discrepancy arises from a different source than the failure to account for hyperparameter uncertainty in the empirical-Bayes estimate. Indeed, even at the extreme, when the empirical-Bayes estimate converges asymptotically to the true variable-inclusion probability, the potential for a serious difference remains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Self-citationStastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,905
Self-citationDeterminants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach
Sala-i-Martin, X., Doppelhofer, G., & Miller, R. I. (2004) · American Economic Review · doi:10.1257/0002828042002570 · Cited by 2,112
Empirical Bayes analysis of a microarray experiment
Efron, B., Tibshirani, R., Storey, J. D., & Tusher, V. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382129 · Cited by 1,733 · cited 2×
“Such justifications can be found for a va? riety of situations in George and Foster (2000), Efron et al. (2001), Johnstone and Silverman (2004), Bogdan, Ghosh and Zak-Szatkowska (2008), Cui and George (2008), Bogdan, Chakrabarti and Ghosh (2008) and Bogdan, Ghosh and Tokdar (2008).”
BackgroundJEFFREYS, H. (1922) · Nature · doi:10.1038/109132a0 · Cited by 1,217
Mixtures of g Priors for Bayesian Variable Selection
Liang, F., Paulo, R., Molina, G., Clyde, M. A., & Berger, J. O. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000001337 · Cited by 1,089
Self-citationModel uncertainty in cross‐country growth regressions
Fernández, C., Ley, E., & Steel, M. F. J. (2001) · Journal of Applied Econometrics · doi:10.1002/jae.623 · Cited by 891
On assessing prior distributions and Bayesian regression analysis with g-prior distributions
Zellner, A. (1986) · Medical Entomology and Zoology · Cited by 886
Optimal predictive model selection
Barbieri, M. M., & Berger, J. O. (2004) · The Annals of Statistics · doi:10.1214/009053604000000238 · Cited by 812
Self-citationA Bayes Rule for the Symmetric Multiple Comparisons Problem
Waller, R. A., & Duncan, D. B. (1969) · Journal of the American Statistical Association · doi:10.1080/01621459.1969.10501073 · Cited by 723
Ockham's Razor and Bayesian Analysis
Jefferys, W. H., & Berger, J. O. (1992) · American Scientist · Cited by 706
Self-citation(1) OPTIMAL PREDICTIVE MODEL SELECTION 1
Barbieri, M. M., Sapienza, U. D. R. “., & Berger, J. O. (2002) · Cited by 640
Self-citationCalibration and empirical Bayes variable selection
George, E. (2000) · Biometrika · doi:10.1093/biomet/87.4.731 · Cited by 520 · cited 7×
“One way is to estimate g by empirical-Bayes methods [George and Foster (2000)]. A second, fully Bayesian, approach involves placing a prior upon g that satisfies the condition ∫ ∞ 0 (1 + g)n−kγ −1π(g)dg = ∞ for all kγ ≤ p, which is a generalization of the condition given in Jeffreys (1961) (see Chapter 5.”
BackgroundMethodInfluentialLey, E., & Steel, M. F. J. (2009) · Journal of Applied Econometrics · doi:10.1002/jae.1057 · Cited by 477
Posterior odds ratios for selected regression hypotheses
Zellner, A., & Siow, A. (1980) · Trabajos de Estadistica Y de Investigacion Operativa · doi:10.1007/bf02888369 · Cited by 463
Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
Johnstone, I. M., & Silverman, B. W. (2004) · The Annals of Statistics · doi:10.1214/009053604000000030 · Cited by 456
Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach
Doppelhofer, G., Miller, R., & Sala-i-Martin, X. (2000) · National Bureau of Economic Research · doi:10.3386/w7750 · Cited by 437
Objective Bayesian Methods for Model Selection: Introduction and Comparison
Berger, J. O., & Pericchi, L. R. (2001) · Institute of Mathematical Statistics Lecture Notes - Monograph Series · doi:10.1214/lnms/1215540968 · Cited by 355
Self-citationAn exploration of aspects of Bayesian multiple testing
Scott, J. G., & Berger, J. O. (2006) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2005.08.031 · Cited by 330
Self-citationA Bayesian perspective on the bonferroni adjustment
Westfall, P. (1997) · Biometrika · doi:10.1093/biomet/84.2.419 · Cited by 316
A Bayes Rule for the Symmetric Multiple Comparisons Problems
Waller, R. A., & Duncan, D. B. (1969) · Journal of the American Statistical Association · doi:10.2307/2286085 · Cited by 233
Group invariance applications in statistics
Eaton, M. L. (1989) · Cited by 192
Bayesian perspectives on multiple comparisons
Berry, D. A., & Hochberg, Y. (1999) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00044-0 · Cited by 171
A Bayesian Mixture Model for Differential Gene Expression
Do, K.-A., Müller, P., & Tang, F. (2005) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2005.05593.x · Cited by 162
BAYES FACTORS AND MARGINAL DISTRIBUTIONS IN INVARIANT SITUATIONS
Pericchi, L. R., & Varshavsky, J. (2016) · Sankhya. Series A · Cited by 154
Empirical Bayes vs. fully Bayes variable selection
Cui, W., & George, E. I. (2007) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2007.02.011 · Cited by 143
Empirical Bayes: Past, Present and Future
Carlin, B. P., & Louis, T. A. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474331 · Cited by 138
Feature-Inclusion Stochastic Search for Gaussian Graphical Models
Scott, J. G., & Carvalho, C. M. (2008) · Journal of Computational and Graphical Statistics · doi:10.1198/106186008x382683 · Cited by 115
Self-citationA comparison of the Benjamini-Hochberg procedure with some Bayesian rules for multiple testing
Bogdan, M., Ghosh, J. K., & Tokdar, S. T. (2008) · Institute of Mathematical Statistics eBooks · doi:10.1214/193940307000000158 · Cited by 110
Objective Bayesian model selection in Gaussian graphical models
Carvalho, C. M., & Scott, J. G. (2009) · Biometrika · doi:10.1093/biomet/asp017 · Cited by 110
Self-citationBayesian Multiple Comparisons Using Dirichlet Process Priors
Gopalan, R., & Berry, D. A. (1998) · Journal of the American Statistical Association · doi:10.1080/01621459.1998.10473774 · Cited by 94
A Bayesian Approach to the Multiplicity Problem for Significance Testing with Binomial Data
Meng, C. Y. K., & Dempster, A. P. (1987) · Biometrics · doi:10.2307/2531814 · Cited by 62
Posterior model probabilities via path‐based pairwise priors
Berger, J. O., & Molina, G. (2005) · Statistica Neerlandica · doi:10.1111/j.1467-9574.2005.00275.x · Cited by 57
Self-citationSelecting explanatory variables with the modified version of the Bayesian information criterion
Bogdan, M., Ghosh, J. K., & Żak‐Szatkowska, M. (2008) · Quality and Reliability Engineering International · doi:10.1002/qre.936 · Cited by 41
Multiple Comparisons, Multiple Tests, and Data Dredging: A Bayesian Perspective
Berry, D. A. (1987) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 29
Nonparametric Bayesian multiple testing for longitudinal performance stratification
Scott, J. G. (2009) · The Annals of Applied Statistics · doi:10.1214/09-aoas252 · Cited by 20
Self-citation
Cited by
… and the Cross-Section of Expected Returns
Harvey, C. R., Liu, Y., & Zhu, H. (2016) · Review of Financial Studies · doi:10.1093/rfs/hhv059 · Cited by 2,119
Bayesian inference for psychology. Part II: Example applications with JASP
Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Selker, R., Gronau, Q. F., Dropmann, D., Boutin, B., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1323-7 · Cited by 1,715
The horseshoe estimator for sparse signals
Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010) · Biometrika · doi:10.1093/biomet/asq017 · Cited by 1,381
Self-citationBobb, J. F., Claus Henn, B., Valeri, L., & Coull, B. A. (2018) · Environmental Health · doi:10.1186/s12940-018-0413-y · Cited by 1,083
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977
Metadata
- Authors
- Scott, J. G., & Berger, J. O.
- Venue
- The Annals of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2010-10-01
- Volume
- 38
- Issue
- 5
- Pages
- 2587-2619
- ISSN
- 0090-5364 (ISSN-L)
- OpenAlex
- W2162888823
- Cited by
- 573
- References
- 36
- FWCI
- 22.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.