The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Sparse multinomial logistic regression: fast algorithms and generalization bounds
Abstract
Recently developed methods for learning sparse classifiers are among the state-of-the-art in supervised learning. These methods learn classifiers that incorporate weighted sums of basis functions with sparsity-promoting priors encouraging the weight estimates to be either significantly large or exactly zero. From a learning-theoretic perspective, these methods control the capacity of the learned classifier by minimizing the number of basis functions used, resulting in better generalization. This paper presents three contributions related to learning sparse classifiers. First, we introduce a true multiclass formulation based on multinomial logistic regression. Second, by combining a bound optimization approach with a component-wise update procedure, we derive fast exact algorithms for learning sparse multiclass classifiers that scale favorably in both the number of training samples and the feature dimensionality, making them applicable even to large data sets in high-dimensional feature spaces. To the best of our knowledge, these are the first algorithms to perform exact multinomial logistic regression with a sparsity-promoting prior. Third, we show how nontrivial generalization bounds can be derived for our classifier in the binary case. Experimental results on standard benchmark data sets attest to the accuracy, sparsity, and efficiency of the proposed methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
A Wavelet Tour of Signal Processing
(1999) · Elsevier eBooks · doi:10.1016/b978-0-12-466606-1.x5000-4 · Cited by 16,227
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (2001) · SIAM Review · doi:10.1137/s003614450037906x · Cited by 5,085
Maximum likelihood estimation from incomplete data via the EM algorithm
Dempster, A. P. (1977) · Journal of the Royal Statistical Society Series A (Statistics in Society) · Cited by 4,543
Cited by
Sparse Representation for Computer Vision and Pattern Recognition
Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T. S., & Yan, S. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2044470 · Cited by 1,847
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning
Sun, Y., Babu, P., & Palomar, D. P. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2601299 · Cited by 1,820
The Group Lasso for Logistic Regression
Meier, L., Van De Geer, S., & Bühlmann, P. (2008) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00627.x · Cited by 1,674
Fisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641
Text Classification Algorithms: A Survey
Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. (2019) · Information · doi:10.3390/info10040150 · Cited by 1,312
Tumor-Associated Macrophages and Survival in Classic Hodgkin's Lymphoma
Steidl, C., Lee, T., Shah, S. P., Farinha, P., Han, G., Nayar, T., Delaney, A., Jones, S. J., Iqbal, J., Weisenburger, D. D., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0905680 · Cited by 1,262
Optimization for Machine Learning
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/8996.001.0001 · Cited by 875
SATzilla: Portfolio-based Algorithm Selection for SAT
Xu, L., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2490 · Cited by 857
Large-Scale Bayesian Logistic Regression for Text Categorization
Genkin, A., Lewis, D. D., & Madigan, D. (2007) · Technometrics · doi:10.1198/004017007000000245 · Cited by 808
Li, J., Bioucas-Dias, J. M., & Plaza, A. (2012) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2162649 · Cited by 731
Optimization with Sparsity-Inducing Penalties
Bach, F., Jenatton, R., Mairal, J., & Obozinski, G. (2025) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000015 · Cited by 730
Advanced Spectral Classifiers for Hyperspectral Images: A review
Ghamisi, P., Plaza, J., Chen, Y., Li, J., & Plaza, A. J. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2616418 · Cited by 675
Optimization with Sparsity-Inducing Penalties
Bach, F. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985118 · Cited by 608
Li, J., Bioucas-Dias, J. M., & Plaza, A. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2060550 · Cited by 594
Hyperspectral Image Classification With Deep Learning Models
Yang, X., Ye, Y., Li, X., Lau, R. Y. K., Zhang, X., & Huang, X. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2815613 · Cited by 576
Hyperspectral Image Classification via Kernel Sparse Representation
Chen, Y., Nasrabadi, N. M., & Tran, T. D. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2012.2201730 · Cited by 514
Generalized Composite Kernel Framework for Hyperspectral Image Classification
Li, J., Marpu, P. R., Plaza, A., Bioucas-Dias, J. M., & Benediktsson, J. A. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2012.2230268 · Cited by 484
PyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI Data
Hanke, M., Halchenko, Y. O., Sederberg, P. B., Hanson, S. J., Haxby, J. V., & Pollmann, S. (2009) · Neuroinformatics · doi:10.1007/s12021-008-9041-y · Cited by 483
Comparison of multivariate classifiers and response normalizations for pattern-information fMRI
Misaki, M., Kim, Y., Bandettini, P. A., & Kriegeskorte, N. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.05.051 · Cited by 458
Deep Few-Shot Learning for Hyperspectral Image Classification
Liu, B., Yu, X., Yu, A., Zhang, P., Wan, G., & Wang, R. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2872830 · Cited by 454
Multiclass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453
Automated Algorithm Selection: Survey and Perspectives
Kerschke, P., Hoos, H. H., Neumann, F., & Trautmann, H. (2019) · Evolutionary Computation · doi:10.1162/evco_a_00242 · Cited by 435
Hyperspectral Image Segmentation Using a New Bayesian Approach With Active Learning
Li, J., Bioucas-Dias, J. M., & Plaza, A. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2128330 · Cited by 417
Computational and experimental identification of novel human imprinted genes
Luedi, P. P., Dietrich, F. S., Weidman, J. R., Bosko, J. M., Jirtle, R. L., & Hartemink, A. J. (2007) · Genome Research · doi:10.1101/gr.6584707 · Cited by 398
Self-citationThe anatomy of spatial neglect
Karnath, H.-O., & Rorden, C. (2012) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2011.06.027 · Cited by 383
Haghighat, M., Abdel-Mottaleb, M., & Alhalabi, W. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2569061 · Cited by 379
Object Class Recognition and Localization Using Sparse Features with Limited Receptive Fields
Mutch, J., & Lowe, D. G. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0118-0 · Cited by 373
A Survey on Spectral–Spatial Classification Techniques Based on Attribute Profiles
Ghamisi, P., Dalla Mura, M., & Benediktsson, J. A. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2358934 · Cited by 350
Chaotic Time Series Prediction Based on a Novel Robust Echo State Network
Li, D., Han, M., & Wang, J. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2188414 · Cited by 341
Li, J., Bioucas-Dias, J. M., & Plaza, A. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2012.2205263 · Cited by 336
Sparse estimation automatically selects voxels relevant for the decoding of fMRI activity patterns
Yamashita, O., Sato, M.-A., Yoshioka, T., Tong, F., & Kamitani, Y. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.05.050 · Cited by 327
Fault diagnosis of low speed bearing based on relevance vector machine and support vector machine
Widodo, A., Kim, E. Y., Son, J.-D., Yang, B.-S., Tan, A. C. C., Gu, D.-S., Choi, B.-K., & Mathew, J. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.09.033 · Cited by 324
Multiple Feature Learning for Hyperspectral Image Classification
Li, J., Huang, X., Gamba, P., Bioucas-Dias, J. M. B., Zhang, L., Benediktsson, J. A., & Plaza, A. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2345739 · Cited by 311
DNA Methylation Dynamics of Human Hematopoietic Stem Cell Differentiation
Farlik, M., Halbritter, F., Müller, F., Choudry, F. A., Ebert, P., Klughammer, J., Farrow, S., Santoro, A., Ciaurro, V., Mathur, A., et al. (2016) · Cell Stem Cell · doi:10.1016/j.stem.2016.10.019 · Cited by 289
Breheny, P., & Huang, J. (2015) · Statistics and Computing · doi:10.1007/s11222-013-9424-2 · Cited by 282
Diagnostic neuroimaging across diseases
Klöppel, S., Abdulkadir, A., Jack, C. R., Koutsouleris, N., Mourão-Miranda, J., & Vemuri, P. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.11.002 · Cited by 279
Deep Air Learning: Interpolation, Prediction, and Feature Analysis of Fine-Grained Air Quality
Qi, Z., Wang, T., Song, G., Hu, W., Li, X., & Zhang, Z. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2823740 · Cited by 274
Sparse logistic regression for whole-brain classification of fMRI data
Ryali, S., Supekar, K., Abrams, D. A., & Menon, V. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.02.040 · Cited by 263
Spectral–Spatial Classification of Hyperspectral Image Based on Deep Auto-Encoder
Ma, X., Wang, H., & Geng, J. (2016) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2016.2517204 · Cited by 258
A Survey on Text Classification Algorithms: From Text to Predictions
Gasparetto, A., Marcuzzo, M., Zangari, A., & Albarelli, A. (2022) · Information · doi:10.3390/info13020083 · Cited by 253
Towards an unconscious neural reinforcement intervention for common fears
Taschereau-Dumouchel, V., Cortese, A., Chiba, T., Knotts, J. D., Kawato, M., & Lau, H. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1721572115 · Cited by 153
Sun, D., van Erp, T. G. M., Thompson, P. M., Bearden, C. E., Daley, M., Kushan, L., Hardt, M. E., Nuechterlein, K. H., Toga, A. W., & Cannon, T. D. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.07.019 · Cited by 148
Chamroukhi, F., Mohammed, S., Trabelsi, D., Oukhellou, L., & Amirat, Y. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.04.003 · Cited by 88
Widodo, A., Yang, B.-S., Kim, E. Y., Tan, A. C. C., & Mathew, J. (2009) · Nondestructive Testing and Evaluation · doi:10.1080/10589750802378974 · Cited by 58
Metadata
- Authors
- Krishnapuram, B., Carin, L., Figueiredo, M. A. T., & Hartemink, A. J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2005-06
- Volume
- 27
- Issue
- 6
- Pages
- 957-968
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2150579376
- Cited by
- 877
- References
- 48
- FWCI
- 24.03
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.