Cited by 110Gaussian Processes and Bayesian InferenceStatistical Methods and InferenceBayesian Modeling and Causal Inference

Predictive automatic relevance determination by expectation propagation

Qi, Yuan (Alan), Minka, Thomas P., Picard, Rosalind W., Ghahramani, Zoubin·2004·Twenty-first international conference on Machine learning - ICML '04

Abstract

In many real-world classification problems the input contains a large number of potentially ir-relevant features. This paper proposes a new Bayesian framework for determining the rele-vance of input features. This approach extends one of the most successful Bayesian methods for feature selection and sparse learning, known as Automatic Relevance Determination (ARD). ARD finds the relevance of features by optimiz-ing the model marginal likelihood, also known as the evidence. We show that this can lead to over-fitting. To address this problem, we propose Pre-dictive ARD based on estimating the predictive performance of the classifier. While the actual leave-one-out predictive performance is generally very costly to compute, the expectation propaga-tion (EP) algorithm proposed by Minka provides an estimate of this predictive performance as a side-effect of its iterations. We exploit this in our algorithm to do feature selection, and to select data points in a sparse Bayesian kernel classifier. Moreover, we provide two other improvements to previous algorithms, by replacing Laplace’s approximation with the generally more accurate EP, and by incorporating the fast optimization algorithm proposed by Faul and Tipping. Our experiments show that our method based on the EP estimate of predictive performance is more accurate on test data than relevance determina-tion by optimizing the evidence.

Integrity signals

Registry

Crossref · 10.1145/1015330.1015418

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Cites

Flagged first, then by citations

1 of the 9 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Qi, Y. (., Minka, T. P., Picard, R. W., & Ghahramani, Z.
Affiliations
  • MIT Media Lab (US)
  • Microsoft Research (United Kingdom) (GB)
Venue
Twenty-first international conference on Machine learning - ICML '04
Publisher
ACM Press
Published
2004
Pages
85
OpenAlex
W2156140776
Cited by
110
References
9
Citations by year
2026: 2 · 2025: 1 · 2024: 3 · 2023: 2 · 2022: 3 · 2021: 3 · 2020: 4 · 2018: 1 · 2017: 2 · 2016: 1 · 2015: 6 · 2014: 3 · 2013: 9 · 2012: 9
FWCI
5.83
Type
conference paper
Language
en
Open access
closed
Locations
  • International Conference on Machine Learning (publishedVersion)
  • Cambridge University Engineering Department Publications Database (submittedVersion)
Funders
  • MIT Media Lab
  • Gatsby Charitable Foundation
  • Defense Advanced Research Projects Agency
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
Keywords
automatic relevance determination, expectation propagation, feature selection, sparse Bayesian learning, overfitting, evidence, Laplace approximation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review