The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Adaptive sparseness for supervised learning
Abstract
The goal of supervised learning is to infer a functional mapping based on a set of training examples. To achieve good generalization, it is necessary to control the "complexity" of the learned function. In Bayesian approaches, this is done by adopting a prior for the parameters of the function being learned. We propose a Bayesian approach to supervised learning, which leads to sparse solutions; that is, in which irrelevant parameters are automatically set exactly to zero. Other ways to obtain sparse classifiers (such as Laplacian priors, support vector machines) involve (hyper)parameters which control the degree of sparseness of the resulting classifiers; these parameters have to be somehow adjusted/estimated from the training data. In contrast, our approach does not involve any (hyper)parameters to be adjusted or estimated. This is achieved by a hierarchical-Bayes interpretation of the Laplacian prior, which is then modified by the adoption of a Jeffreys' noninformative hyperprior. Implementation is carried out by an expectation-maximization (EM) algorithm. Experiments with several benchmark data sets show that the proposed approach yields state-of-the-art performance. In particular, our method outperforms SVMs and performs competitively with the best alternative techniques, although it involves no tuning or adjustment of sparseness-controlling hyperparameters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
McCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Ideal spatial adaptation by wavelet shrinkage
Donoho, D. L., & Johnstone, I. M. (1994) · Biometrika · doi:10.1093/biomet/81.3.425 · Cited by 8,760
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Pattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,459
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (2001) · SIAM Review · doi:10.1137/s003614450037906x · Cited by 5,085
Spline Models for Observational Data.
Muller, H.-G., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,928
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Spline Models for Observational Data
Wahba, G. (1990) · doi:10.1137/1.9781611970128 · Cited by 4,184
Networks for approximation and learning
Poggio, T., & Girosi, F. (1990) · Proceedings of the IEEE · doi:10.1109/5.58326 · Cited by 3,252
Bayesian Analysis of Binary and Polychotomous Response Data
Albert, J. H., & Chib, S. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476321 · Cited by 3,167
Schuenemeyer, J. H., McCullagh, P., & Nelder, J. A. (1992) · Technometrics · doi:10.2307/1269238 · Cited by 2,667
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.2307/1267351 · Cited by 1,955
Using the Nyström Method to Speed Up Kernel Machines
Williams, C. K. I., & Seeger, M. (2000) · Cited by 1,646
Multivariate Statistical Modelling Based on Generalized Linear Models
Fahrmeir, L., & Tutz, G. (1994) · Springer Series in Statistics · doi:10.1007/978-1-4899-0010-4 · Cited by 1,391
Tipping, M. E. (1999) · Cited by 912
A new approach to variable selection in least squares problems
Osborne, M. (2000) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/20.3.389 · Cited by 859
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.1080/00401706.2000.10485983 · Cited by 808
Bayesian classification with Gaussian processes
Williams, C. K. I., & Barber, D. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735807 · Cited by 766
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
Williams, C. K. I. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_23 · Cited by 659
Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors
Moulin, P., & Liu, J. (1999) · IEEE Transactions on Information Theory · doi:10.1109/18.761332 · Cited by 500
Berger, J. O. (1980) · Springer Series in Statistics · doi:10.1007/978-1-4757-1727-3 · Cited by 451
Bayesian Regularization and Pruning Using a Laplace Prior
Williams, P. M. (1995) · Neural Computation · doi:10.1162/neco.1995.7.1.117 · Cited by 377
Variational Relevance Vector Machines
Bishop, C., & Tipping, M. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3838 · Cited by 352
Normal/Independent Distributions and Their Applications in Robust Regression
Lange, K., & Sinsheimer, J. S. (1993) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1993.10474606 · Cited by 322
Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior
Figueiredo, M. A. T., & Nowak, R. D. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.941856 · Cited by 235
Self-citationOn Different Facets of Regularization Theory
Chen, Z., & Haykin, S. (2002) · Neural Computation · doi:10.1162/089976602760805296 · Cited by 148
Bayesian Non-Linear Modeling for the Prediction Competition
Mackay, D. (1996) · Springer eBooks · doi:10.1007/978-94-015-8729-7_18 · Cited by 143
田中, 豊. (1995) · 日本統計学会誌 · Cited by 136
Adaptive Sparseness Using Jeffreys' Prior
Figueiredo, M. A. T. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0094 · Cited by 129
Self-citationNormal/Independent Distributions and Their Applications in Robust Regression
Lange, K. L., & Sinsheimer, J. S. (1993) · Journal of Computational and Graphical Statistics · doi:10.2307/1390698 · Cited by 109
Least Absolute Shrinkage is Equivalent to Quadratic Penalization
Grandvalet, Y. (1998) · Perspectives in neural computing · doi:10.1007/978-1-4471-1599-1_27 · Cited by 104
Seeger, M. (1999) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 104
Bayesian learning of sparse classifiers
Figueiredo, M. A. T., & Jain, A. K. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2001.990453 · Cited by 65
Self-citationFigueiredo, M. A. T. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2000.906151 · Cited by 42
Self-citation
Cited by
Park, T., & Casella, G. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000337 · Cited by 2,934
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration
Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.909319 · Cited by 2,064
Self-citationThe horseshoe estimator for sparse signals
Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010) · Biometrika · doi:10.1093/biomet/asq017 · Cited by 1,381
Sparse multinomial logistic regression: fast algorithms and generalization bounds
Krishnapuram, B., Carin, L., Figueiredo, M. A. T., & Hartemink, A. J. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.127 · Cited by 877
Self-citationLarge-Scale Bayesian Logistic Regression for Text Categorization
Genkin, A., Lewis, D. D., & Madigan, D. (2007) · Technometrics · doi:10.1198/004017007000000245 · Cited by 808
Metadata
- Authors
- Figueiredo, M. A. T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2003-09
- Volume
- 25
- Issue
- 9
- Pages
- 1150-1159
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2146571341
- Cited by
- 536
- References
- 43
- FWCI
- 15.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.