The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Adaptive sparseness for supervised learning
Abstract
The goal of supervised learning is to infer a functional mapping based on a set of training examples. To achieve good generalization, it is necessary to control the "complexity" of the learned function. In Bayesian approaches, this is done by adopting a prior for the parameters of the function being learned. We propose a Bayesian approach to supervised learning, which leads to sparse solutions; that is, in which irrelevant parameters are automatically set exactly to zero. Other ways to obtain sparse classifiers (such as Laplacian priors, support vector machines) involve (hyper)parameters which control the degree of sparseness of the resulting classifiers; these parameters have to be somehow adjusted/estimated from the training data. In contrast, our approach does not involve any (hyper)parameters to be adjusted or estimated. This is achieved by a hierarchical-Bayes interpretation of the Laplacian prior, which is then modified by the adoption of a Jeffreys' noninformative hyperprior. Implementation is carried out by an expectation-maximization (EM) algorithm. Experiments with several benchmark data sets show that the proposed approach yields state-of-the-art performance. In particular, our method outperforms SVMs and performs competitively with the best alternative techniques, although it involves no tuning or adjustment of sparseness-controlling hyperparameters.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
McCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
Ideal spatial adaptation by wavelet shrinkage
Donoho, D. L., & Johnstone, I. M. (1994) · Biometrika · doi:10.1093/biomet/81.3.425 · Cited by 8,760
Cited by
Park, T., & Casella, G. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000337 · Cited by 2,934
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration
Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.909319 · Cited by 2,064
Self-citationThe horseshoe estimator for sparse signals
Carvalho, C. M., Polson, N. G., & Scott, J. G. (2010) · Biometrika · doi:10.1093/biomet/asq017 · Cited by 1,381
Sparse multinomial logistic regression: fast algorithms and generalization bounds
Krishnapuram, B., Carin, L., Figueiredo, M. A. T., & Hartemink, A. J. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.127 · Cited by 877
Self-citationLarge-Scale Bayesian Logistic Regression for Text Categorization
Genkin, A., Lewis, D. D., & Madigan, D. (2007) · Technometrics · doi:10.1198/004017007000000245 · Cited by 808
Bayesian Compressive Sensing Using Laplace Priors
Babacan, S. D., Molina, R., & Katsaggelos, A. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2032894 · Cited by 804
A review of Bayesian variable selection methods: what, how and which
O'Hara, R. B., & Sillanpää, M. J. (2009) · Bayesian Analysis · doi:10.1214/09-ba403 · Cited by 771
A Model of Text for Experimentation in the Social Sciences
Roberts, M. E., Stewart, B. M., & Airoldi, E. M. (2016) · Journal of the American Statistical Association · doi:10.1080/01621459.2016.1141684 · Cited by 728
Yardibi, T., Li, J., Stoica, P., Xue, M., & Baggeroer, A. B. (2010) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2010.5417172 · Cited by 699
Penalized regression, standard errors, and Bayesian lassos
Casella, G., Ghosh, M., Gill, J., & Kyung, M. (2010) · Bayesian Analysis · doi:10.1214/10-ba607 · Cited by 567
Inference with normal-gamma prior distributions in regression problems
Brown, P. J., & Griffin, J. E. (2010) · Bayesian Analysis · doi:10.1214/10-ba507 · Cited by 462
Multiclass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453
Iterative Reweighted $\ell_1$ and $\ell_2$ Methods for Finding Sparse Solutions
Wipf, D., & Nagarajan, S. (2010) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2010.2042413 · Cited by 448
Surface EMG in advanced hand prosthetics
Castellini, C., & van der Smagt, P. (2009) · Biological Cybernetics · doi:10.1007/s00422-008-0278-1 · Cited by 429
Methodological implementation of mixed linear models in multi-locus genome-wide association studies
Wen, Y.-J., Zhang, H., Ni, Y.-L., Huang, B., Zhang, J., Feng, J.-Y., Wang, S.-B., Dunwell, J. M., Zhang, Y.-M., & Wu, R. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbw145 · Cited by 394
Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction*
Polson, N. G., & Scott, J. G. (2011) · Bayesian Statistics 9 · doi:10.1093/acprof:oso/9780199694587.003.0017 · Cited by 387
Object Class Recognition and Localization Using Sparse Features with Limited Receptive Fields
Mutch, J., & Lowe, D. G. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0118-0 · Cited by 373
Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches
Schmidt, M., Fung, G., & Rosales, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74958-5_28 · Cited by 353
Bayesian LASSO for Quantitative Trait Loci Mapping
Yi, N., & Xu, S. (2008) · Genetics · doi:10.1534/genetics.107.085589 · Cited by 344
Kadaoluwa Pathirannahalage, S. P., Meftahi, N., Elbourne, A., Weiss, A. C. G., McConville, C. F., Padua, A., Winkler, D. A., Costa Gomes, M., Greaves, T. L., Le, T. C., et al. (2021) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.1c00794 · Cited by 302
Sparse Additive Generative Models of Text
Eisenstein, J., Ahmed, A. A., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6476342 · Cited by 277
Bayesian Inference and Optimal Design for the Sparse Linear Model
Seeger, M. (2008) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.5555/1390681.1390707 · Cited by 277
Sparse Bayesian Classification of EEG for Brain–Computer Interface
Zhang, Y., Zhou, G., Jin, J., Zhao, Q., Wang, X., & Cichocki, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2476656 · Cited by 264
Li, Z., & Sillanpää, M. J. (2012) · Theoretical and Applied Genetics · doi:10.1007/s00122-012-1892-9 · Cited by 258
Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark
Lüthen, N., Marelli, S., & Sudret, B. (2021) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/20m1315774 · Cited by 256
A renaissance of neural networks in drug discovery
Baskin, I. I., Winkler, D., & Tetko, I. V. (2016) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2016.1201262 · Cited by 235
Variational Bayesian Sparse Kernel-Based Blind Image Deconvolution With Student's-t Priors
Tzikas, D. G., Likas, A. C., & Galatsanos, N. P. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2011757 · Cited by 80
Relevance Vector Machines: Sparse Classification Methods for QSAR
Burden, F. R., & Winkler, D. A. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00261 · Cited by 51
Mohsenzadeh, Y., Sheikhzadeh, H., & Nazari, S. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.06.028 · Cited by 22
Huh, Y. H., Noh, M., Burden, F. R., Chen, J. C., Winkler, D. A., & Sherley, J. L. (2015) · Stem Cell Research · doi:10.1016/j.scr.2014.12.007 · Cited by 20
“Figueiredo reported an EM algorithm using a sparse (noninformative) prior that provided parameter-free adaptive sparseness methodology (Figueiredo, 2003).”
Method
Metadata
- Authors
- Figueiredo, M. A. T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2003-09
- Volume
- 25
- Issue
- 9
- Pages
- 1150-1159
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2146571341
- Cited by
- 536
- References
- 43
- FWCI
- 15.19
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.