CoChat
Sign Up
Cited by 113Statistical Methods and InferenceAdvanced Statistical Methods and ModelsStatistical Methods and Applications

Stability

Open access
Yu, Bin·2013·Bernoulli
DOI 10.3150/13-bejsp14
Open this paper in CoChat — free accountOpen PDF

Abstract

Reproducibility is imperative for any scientific discovery. More often than not, modern scientific findings rely on statistical analysis of high-dimensional data. At a minimum, reproducibility manifests itself in stability of statistical results relative to “reasonable” perturbations to data and to the model used. Jacknife, bootstrap, and cross-validation are based on perturbations to data, while robust statistics methods deal with perturbations to models. In this article, a case is made for the importance of stability in statistics. Firstly, we motivate the necessity of stability for interpretable and reliable encoding models from brain fMRI signals. Secondly, we find strong evidence in the literature to demonstrate the central role of stability in statistical inference, such as sensitivity analysis and effect detection. Thirdly, a smoothing parameter selector based on estimation stability (ES), ES-CV, is proposed for Lasso, in order to bring stability to bear on cross-validation (CV). ES-CV is then utilized in the encoding models to reduce the number of predictors by 60% with almost no loss (1.3%) of prediction performance across over 2,000 voxels. Last, a novel “stability” argument is seen to drive new results that shed light on the intriguing interactions between sample to sample variability and heavier tail error distribution (e.g., double-exponential) in high-dimensional regression models with $p$ predictors and $n$ independent samples. In particular, when $p/n\rightarrow\kappa\in(0.3,1)$ and the error distribution is double-exponential, the Ordinary Least Squares (OLS) is a better estimator than the Least Absolute Deviation (LAD) estimator.

Integrity signals

Registry

Crossref · 10.3150/13-bejsp14

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

1 of the 70 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Yu, B.
Affiliations
University of California, Berkeley (US)
Corresponding author
Yu, Bin
ORCID
Yu, Bin
Venue
Bernoulli
Publisher
Bernoulli Society for Mathematical Statistics and Probability
Published
2013-09-01
Volume
19
Issue
4
Pages
1484-1500
ISSN
1350-7265 (ISSN-L)
DOI
10.3150/13-bejsp14
OpenAlex
W4230605228
Cited by
113
References
70
Citations by year
2026: 5 · 2025: 12 · 2024: 14 · 2023: 9 · 2022: 9 · 2021: 16 · 2020: 12 · 2019: 7 · 2018: 8 · 2017: 9 · 2016: 3 · 2015: 8 · 2014: 1
FWCI
3.53
Type
journal article
Language
en
Open access
Locations
  • Bernoulli (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Project Euclid (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (CCF0-939370, 0939370, 1107000, 0835531)
  • Army Research Office (W911NF-11-1- 0114, W911NF, W911NF-11)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
Keywords
estimation stability, reproducibility, cross-validation, jackknife, bootstrapping, robust statistics, sensitivity analysis, high-dimensional regression, sample-to-sample variation, least absolute deviation, ordinary least squares, feature selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review