The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Distributed Sparse Linear Regression
Open accessAbstract
The Lasso is a popular technique for joint estimation and continuous variable selection, especially well-suited for sparse and possibly under-determined linear regression problems. This paper develops algorithms to estimate the regression coefficients via Lasso when the training data are distributed across different agents, and their communication to a central processing unit is prohibited for e.g., communication cost or privacy reasons. A motivating application is explored in the context of wireless communications, whereby sensing cognitive radios collaborate to estimate the radio-frequency power spectrum density. Attaining different tradeoffs between complexity and convergence speed, three novel algorithms are obtained after reformulating the Lasso into a separable form, which is iteratively minimized using the alternating-direction method of multipliers so as to gain the desired degree of parallelization. Interestingly, the per agent estimate updates are given by simple soft-thresholding operations, and inter-agent communication overhead remains at affordable level. Without exchanging elements from the different training sets, the local estimates consent to the global Lasso solution, i.e., the fit that would be obtained if the entire data set were centrally available. Numerical experiments with both simulated and real data demonstrate the merits of the proposed distributed schemes, corroborating their convergence and global optimality. The ideas in this paper can be easily extended for the purpose of fitting related models in a distributed fashion, including the adaptive Lasso, elastic net, fused Lasso and nonnegative garrote.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
Wireless sensor networks: a survey
Akyildiz, I. F., Su, W., Sankarasubramaniam, Y., & Cayirci, E. (2002) · Computer Networks · doi:10.1016/s1389-1286(01)00302-4 · Cited by 17,418
Consensus and Cooperation in Networked Multi-Agent Systems
Olfati-Saber, R., Fax, J. A., & Murray, R. M. (2007) · Proceedings of the IEEE · doi:10.1109/jproc.2006.887293 · Cited by 10,522 · cited 1×
“Tutorial treatments of related consensus-based distributed approaches can be found in [9] and [23].”
BackgroundEfron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Fan, J., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,900
Coordination of groups of mobile autonomous agents using nearest neighbor rules
Jadbabaie, A., Lin, J., & Morse, A. S. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.812781 · Cited by 8,339
The adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
The Split Bregman Method for L1-Regularized Problems
Goldstein, T., & Osher, S. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080725891 · Cited by 4,282
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Sparsity and Smoothness Via the Fused Lasso
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., & Knight, K. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00490.x · Cited by 2,724
Fast linear iterations for distributed averaging
Xiao, L., & Boyd, S. (2004) · Systems & Control Letters · doi:10.1016/j.sysconle.2004.02.022 · Cited by 2,706
A dual algorithm for the solution of nonlinear variational problems via finite element approximation
Gabay, D., & Mercier, B. (1976) · Computers & Mathematics with Applications · doi:10.1016/0898-1221(76)90003-1 · Cited by 2,514
Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization
Tseng, P. (2001) · Journal of Optimization Theory and Applications · doi:10.1023/a:1017501703105 · Cited by 1,989
Pathwise coordinate optimization
Friedman, J., Hastie, T., Höfling, H., & Tibshirani, R. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas131 · Cited by 1,895
Sparse Reconstruction by Separable Approximation
Wright, S. J., Nowak, R. D., & Figueiredo, M. A. T. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2016892 · Cited by 1,854
Parlar, M. (2000) · Interactive Operations Research with Maple · doi:10.1007/978-1-4612-1356-7_5 · Cited by 1,741
Glowinski, R., & Marroco, A. (1975) · Revue française d'automatique, informatique, recherche opérationnelle. Analyse numérique · doi:10.1051/m2an/197509r200411 · Cited by 1,105
Better Subset Regression Using the Nonnegative Garrote
Breiman, L. (1995) · Technometrics · doi:10.1080/00401706.1995.10484371 · Cited by 1,043
Tools for privacy preserving distributed data mining
Clifton, C., Kantarcioglu, M., Vaidya, J., Lin, X., & Zhu, M. Y. (2002) · ACM SIGKDD Explorations Newsletter · doi:10.1145/772862.772867 · Cited by 890
Stamey, T. A., Kabalin, J. N., Mcneal, J. E., Johnstone, I. M., Freiha, F., Redwine, E. A., & Yang, N. (1989) · Journal of Urology · doi:10.1016/s0022-5347(17)41175-x · Cited by 871
Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization
Sundhar Ram, S., Nedić, A., & Veeravalli, V. V. (2010) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-010-9737-7 · Cited by 814 · cited 4×
“DCD-Lasso and D-Lasso are also compared with the distributed sub-gradient methods in [28] and [22], for which equal neighbor combining weights [22], zero initial conditions, and a diminishing stepsize are adopted.”
BackgroundMethodInfluentialCoordinate descent algorithms for lasso penalized regressionPreprint
Wu, T. T., & Lange, K. L. (2008) · Cited by 735
Gossip Algorithms for Distributed Signal Processing
Dimakis, A. G., Kar, S., Moura, J. M. F., Rabbat, M., & Scaglione, A. (2018) · doi:10.1184/r1/6468842.v1 · Cited by 692
Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals
Schizas, I. D., Ribeiro, A., & Giannakis, G. B. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.906734 · Cited by 685
Self-citationIncremental Subgradient Methods for Nondifferentiable Optimization
Nedic, A., & Bertsekas, D. P. (2001) · SIAM Journal on Optimization · doi:10.1137/s1052623499362111 · Cited by 589
Coordinate descent algorithms for lasso penalized regression
Wu, T. T., & Lange, K. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas147 · Cited by 563
Distributed Spectrum Sensing for Cognitive Radio Networks by Exploiting Sparsity
Bazerque, J. A., & Giannakis, G. B. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2038417 · Cited by 510
Self-citationQuantized incremental algorithms for distributed optimization
Rabbat, M. G., & Nowak, R. D. (2005) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2005.843546 · Cited by 408
Consensus-based distributed support vector machines
Forero, P. A., Caño, A., & Giannakis, G. B. (2010) · Cited by 364
Self-citationFast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches
Schmidt, M., Fung, G., & Rosales, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74958-5_28 · Cited by 353
Consensus-based distributed linear support vector machines
Forero, P. A., Cano, A., & Giannakis, G. B. (2010) · Proceedings of the 9th ACM/IEEE International Conference on Information Processing in Sensor Networks · doi:10.1145/1791212.1791218 · Cited by 171
Self-citationA Collaborative Training Algorithm for Distributed Learning
Predd, J. B., Kulkarni, S. R., & Poor, H. V. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2012992 · Cited by 145
Generalized consensus computation in networked systems with erasure links
Rabbat, M., Nowak, R. D., & Bucklew, J. A. (2005) · IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2005.1506308 · Cited by 110
Distributed In-Network Channel Decoding
Zhu, H., Giannakis, G. B., & Cano, A. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2023936 · Cited by 81
Self-citationStochastic Incremental Gradient Descent for Estimation in Sensor Networks
Ram, S. S., Nedić, A., & Veeravalli, V. V. (2007) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2007.4487280 · Cited by 66
Secondary spectrum access networks
Hong, X., Wang, C., Chen, H., & Zhang, K. (2009) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2009.932543 · Cited by 57
Cooperative sensing for cognitive radio using decentralized projection algorithms
Barbarossa, S., Scutari, G., & Battisti, T. (2009) · IEEE Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2009.5161758 · Cited by 11
Cited by
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
Shi, W., Ling, Q., Wu, G., & Yin, W. (2015) · SIAM Journal on Optimization · doi:10.1137/14096668x · Cited by 1,261
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs
Nedić, A., Olshevsky, A., & Shi, W. (2017) · SIAM Journal on Optimization · doi:10.1137/16m1084316 · Cited by 1,069
“The resulting algorithms include distributed dual decomposition [65] and decentralized alternating direction method of multipliers (ADMM) [5, 32].”
MethodOn the Linear Convergence of the ADMM in Decentralized Consensus Optimization
Shi, W., Ling, Q., Yuan, K., Wu, G., & Yin, W. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2304432 · Cited by 820
Metadata
- Authors
- Mateos, G., Bazerque, J. A., & Giannakis, G. B.
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2010-10
- Volume
- 58
- Issue
- 10
- Pages
- 5262-5276
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2151565294
- Cited by
- 550
- References
- 43
- FWCI
- 15.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.