Model Selection and Estimation in Regression with Grouped Variables
Open accessAbstract
Summary: We consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor analysis-of-variance problem as the most important and well-known example. Instead of selecting factors by stepwise backward elimination, we focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection. The lasso, the LARS algorithm and the non-negative garrotte are recently proposed regression methods that can be used to select individual variables. We study and propose efficient algorithms for the extensions of these methods for factor selection and show that these extensions give superior performance to the traditional stepwise backward elimination method in factor selection problems. We study the similarities and the differences between these methods. Simulations and real examples are used to illustrate the methods.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
New filtering formulations that organize streamlines into groups and seek the minimum number of bundles using Group Lasso significantly improve anatomical accuracy of the connectome.
“This prior knowledge helps resolve some of the ambiguities present in the data and, as a consequence, allows significant improvement of the anatomical accuracy of the connectome (Ocampo-Pineda et al., 2021; Schiavi et al., 2020a).”
From Quantitative mapping of the brain’s structural connectivity using diffusion MRI tractography: A review (2022) · cites this paper as Yuan and Lin, 2006
Section 2 (Group lasso; page not available) says group lasso selects groups of variables and encourages sparsity at the factor level. The paper does not address streamline filtering, bundles, connectomes, or improved anatomical accuracy; those application and outcome claims are unsupported by this source. (LLM, conf=0.84)
Experimental — accuracy not yet measuredGroup Lasso helps in selecting physically meaningful groups of attributes in data science models.
“Group Lasso helps in selecting physically meaningful groups of attributes in the data science models, and various extensions of group Lasso have been explored for handling different types of domain characteristics, e.g., overlapping group Lasso [63], tree-guided group Lasso [64], and sparse group Lasso [65].”
From Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2017) · cites this paper as [62]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305 · see citing works
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357 · see citing works
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
B, J. F. A., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,919 · see citing works
Kupper, L. L., Hosmer, D. W., & Lemeshow, S. A. (1990) · Journal of the American Statistical Association · doi:10.2307/2290035 · Cited by 7,690 · see citing works
Variable Selection via Gibbs Sampling
George, E. I., & McCulloch, R. E. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476353 · Cited by 2,828 · see citing works
Penalized Regressions: The Bridge versus the Lasso
Fu, W. J. (1998) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1998.10474784 · Cited by 1,096 · see citing works
Better Subset Regression Using the Nonnegative Garrote
Breiman, L. (1995) · Technometrics · doi:10.2307/1269730 · Cited by 1,045 · see citing works
Component selection and smoothing in multivariate nonparametric regression
Lin, Y., & Zhang, H. H. (2006) · The Annals of Statistics · doi:10.1214/009053606000000722 · Cited by 550 · see citing works
Self-citationThe Risk Inflation Criterion for Multiple Regression
Foster, D. P., & George, E. I. (1994) · The Annals of Statistics · doi:10.1214/aos/1176325766 · Cited by 542 · see citing works
Calibration and empirical Bayes variable selection
George, E. (2000) · Biometrika · doi:10.1093/biomet/87.4.731 · Cited by 520 · see citing works
Piecewise linear regularized solution paths
Rosset, S., & Zhu, J. (2007) · The Annals of Statistics · doi:10.1214/009053606000001370 · Cited by 484 · see citing works
The Variable Selection Problem
George, E. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474336 · Cited by 385 · see citing works
Edrei, A., Saff, E. B., & Varga, R. S. (1983) · Lecture notes in mathematics · doi:10.1007/bfb0070484 · Cited by 276 · see citing works
Shen, X. T., & Ye, J. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502753479356 · Cited by 197 · see citing works
Adaptive regression and model selection in data mining problems
Bakin, S. (1999) · ANU Open Research (Australian National University) · doi:10.25911/5d78db4c25dbb · Cited by 177 · see citing works
Edrei, A., Saff, E. B., & Varga, R. S. (1983) · Lecture notes in mathematics · doi:10.1007/bfb0070489 · Cited by 123 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Parikh, N., & Boyd, S. (2014) · Foundations and Trends® in Optimization · doi:10.1561/2400000003 · Cited by 4,102 · see citing works
Feature Selection: A Data Perspective
Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2018) · ACM Computing Surveys · doi:10.1145/3136625 · Cited by 2,442 · see citing works
Wisdom of crowds for robust gene network inference
The DREAM5 Consortium, Marbach, D., Costello, J. C., Küffner, R., Vega, N. M., Prill, R. J., Camacho, D. M., Allison, K. R., Kellis, M., Collins, J. J., et al. (2012) · Nature Methods · doi:10.1038/nmeth.2016 · Cited by 1,667 · see citing works
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S., Samatova, N., & Kumar, V. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2720168 · Cited by 1,752 · see citing works
Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2013) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2012.681250 · Cited by 1,380 · see citing works
Dissecting Characteristics Nonparametrically
Freyberger, J., Neuhierl, A., & Weber, M. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhz123 · Cited by 619 · see citing works
Ogutu, J. O., Schulz-Streeck, T., & Piepho, H.-P. (2012) · BMC Proceedings · doi:10.1186/1753-6561-6-s2-s10 · Cited by 440 · see citing works
Recent advances in feature selection and its applications
Li, Y., Li, T., & Liu, H. (2017) · Knowledge and Information Systems · doi:10.1007/s10115-017-1059-8 · Cited by 400 · see citing works
Learning generative models for protein fold families
Balakrishnan, S., Kamisetty, H., Carbonell, J. G., Lee, S., & Langmead, C. J. (2011) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.22934 · Cited by 316 · see citing works
Zhang, F., Daducci, A., He, Y., Schiavi, S., Seguin, C., Smith, R. E., Yeh, C.-H., Zhao, T., & O’Donnell, L. J. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2021.118870 · Cited by 302 · see citing works
“using the Group Lasso regularization [525] to promote sparsity”
Tutorial in biostatistics: data‐driven subgroup identification and analysis in clinical trials
Lipkovich, I., Dmitrienko, A., & B., R. (2017) · Statistics in Medicine · doi:10.1002/sim.7064 · Cited by 291 · see citing works
Zhu, X., Suk, H.-I., Wang, L., Lee, S.-W., & Shen, D. (2017) · Medical Image Analysis · doi:10.1016/j.media.2015.10.008 · Cited by 224 · see citing works
Peng, J., Zhu, J., Bergamaschi, A., Han, W., Noh, D.-Y., Pollack, J. R., & Wang, P. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas271 · Cited by 211 · see citing works
Zhu, X., Suk, H.-I., & Shen, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.05.078 · Cited by 202 · see citing works
General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models
Molnar, C., König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C. A., Casalicchio, G., Grosse-Wentrup, M., & Bischl, B. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_4 · Cited by 211 · see citing works
A Logistic Normal Multinomial Regression Model for Microbiome Compositional Data Analysis
Xia, F., Chen, J., Fung, W. K., & Li, H. (2013) · Biometrics · doi:10.1111/biom.12079 · Cited by 144 · see citing works
Association screening of common and rare genetic variants by penalized regression
Zhou, H., Sehl, M. E., Sinsheimer, J. S., & Lange, K. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq448 · Cited by 137 · see citing works
Multi-dimensional coding by basolateral amygdala neurons
Kyriazi, P., Headley, D. B., & Pare, D. (2018) · Neuron · doi:10.1016/j.neuron.2018.07.036 · Cited by 130 · see citing works
“Importantly, this type of GLM allows for dimensionality reduction in correlated data and encourages sparsity when identifying the task variables that are related to cell activity (Breheny and Huang, 2015; Tibshirani, 1996; Yuan and Lin, 2006).”
Combining sparsity and rotational invariance in EEG/MEG source reconstruction
Haufe, S., Nikulin, V. V., Ziehe, A., Müller, K.-R., & Nolte, G. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.04.246 · Cited by 130 · see citing works
Network inference with Granger causality ensembles on single-cell transcriptomics
Deshpande, A., Chu, L.-F., Stewart, R., & Gitter, A. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.110333 · Cited by 130 · see citing works
Group lasso regularized deep learning for cancer prognosis from multi-omics and clinical features
Xie, G., Dong, C., Kong, Y., Zhong, J. F., Li, M., & Wang, K. (2019) · Genes · doi:10.3390/genes10030240 · Cited by 102 · see citing works
Tuia, D., Flamary, R., & Courty, N. (2015) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2015.01.006 · Cited by 102 · see citing works
Initiative, T. A. D. N., Zu, C., Jie, B., Liu, M., Chen, S., Shen, D., & Zhang, D. (2016) · Brain Imaging and Behavior · doi:10.1007/s11682-015-9480-7 · Cited by 95 · see citing works
A survey on online feature selection with streaming features
Hu, X., Zhou, P., Li, P., Wang, J., & Wu, X. (2018) · Frontiers of Computer Science · doi:10.1007/s11704-016-5489-3 · Cited by 88 · see citing works
Guo, H., Chen, L., Peng, L., & Chen, G. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971708 · Cited by 83 · see citing works
Global Model Interpretation Via Recursive Partitioning
Yang, C., Rangarajan, A., & Ranka, S. (2018) · 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc/smartcity/dss.2018.00256 · Cited by 93 · see citing works
Space–time event sparse penalization for magneto-/electroencephalography
Bolstad, A., Veen, B. V., & Nowak, R. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.01.056 · Cited by 71 · see citing works
Sparse Group Lasso: Consistency and Climate Applications
Chatterjee, S., Steinhaeuser, K., Banerjee, A., Chatterjee, S., & Ganguly, A. (2012) · Proceedings of the 2012 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972825.5 · Cited by 72 · see citing works
Model Selection of the Effect of Binary Exposures over the Life Course
Smith, A. D. A. C., Heron, J., Mishra, G., Gilthorpe, M. S., Ben-Shlomo, Y., & Tilling, K. (2015) · Epidemiology · doi:10.1097/ede.0000000000000348 · Cited by 63 · see citing works
Group additive regression models for genomic data analysis
Luan, Y., & Li, H. (2008) · Biostatistics · doi:10.1093/biostatistics/kxm015 · Cited by 48 · see citing works
Segmentation of biological multivariate time-series data
Omranian, N., Mueller-Roeber, B., & Nikoloski, Z. (2015) · Scientific Reports · doi:10.1038/srep08937 · Cited by 28 · see citing works
Sparse Reduced-Rank Regression for Exploratory Visualisation of Paired Multivariate Data
Kobak, D., Bernaerts, Y., Weis, M. A., Scala, F., Tolias, A. S., & Berens, P. (2021) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12494 · Cited by 24 · see citing works
A group lasso based sparse KNN classifier
Zheng, S., & Ding, C. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.12.020 · Cited by 22 · see citing works
A new multiple kernel-based regularization method for identification of delay linear dynamic systems
Chen, X., Mao, Z., & Jia, R. (2020) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2020.103971 · Cited by 5 · see citing works
- Authors
- Yuan, M., & Lin, Y.
- Affiliations
- Georgia Institute of Technology (US)
- University of Wisconsin–Madison (US)
- Corresponding author
- Yuan, Ming
- ORCID
- Venue
- Journal of the Royal Statistical Society Series B: Statistical Methodology
- Publisher
- Oxford University Press (OUP)
- Published
- 2005-12-21
- Volume
- 68
- Issue
- 1
- Pages
- 49-67
- ISSN
- 1369-7412 (ISSN-L)
- 1467-9868
- OpenAlex
- W2138019504
- Cited by
- 7,261
- References
- 17
- Citations by year
- 2026: 223 · 2025: 312 · 2024: 376 · 2023: 361 · 2022: 407 · 2021: 559 · 2020: 469 · 2019: 571 · 2018: 479 · 2017: 483 · 2016: 505 · 2015: 468 · 2014: 495 · 2013: 428 · 2012: 387
- FWCI
- 39.01
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the Royal Statistical Society Series B (Statistical Methodology) (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation (DMS-0134987)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- factor selection, LASSO, sparse regression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
