The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem
Abstract
Given a large overcomplete dictionary of basis vectors, the goal is to simultaneously represent L>1 signal vectors using coefficient expansions marked by a common sparsity profile. This generalizes the standard sparse representation problem to the case where multiple responses exist that were putatively generated by the same small subset of features. Ideally, the associated sparse generating weights should be recovered, which can have physical significance in many applications (e.g., source localization). The generic solution to this problem is intractable and, therefore, approximate procedures are sought. Based on the concept of automatic relevance determination, this paper uses an empirical Bayesian prior to estimate a convenient posterior distribution over candidate basis vectors. This particular approximation enforces a common sparsity profile and consistently places its prominent posterior mass on the appropriate region of weight-space necessary for simultaneous sparse recovery. The resultant algorithm is then compared with multiple response extensions of matching pursuit, basis pursuit, FOCUSS, and Jeffreys prior-based Bayesian methods, finding that it often outperforms the others. Additional motivation for this particular choice of cost function is also provided, including the analysis of global and local minima and a variational derivation that highlights the similarities and differences between the proposed algorithm and previous approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Donoho, D. L. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.871582 · Cited by 23,994
Donoho, D. L. (2004) · Cited by 16,703
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Applied Multivariate Statistical Analysis.
Johnson, A. R., & Wichern, D. W. (1988) · Biometrics · doi:10.2307/2531616 · Cited by 11,353
Fundamentals of Statistical Signal Processing: Estimation Theory
Sengijpta, S. K. (1995) · Technometrics · doi:10.1080/00401706.1995.10484391 · Cited by 11,076
Rockafellar, R. T. (1970) · Princeton University Press eBooks · doi:10.1515/9781400873173 · Cited by 10,801
Matching pursuits with time-frequency dictionaries
Mallat, S. G., & Zhang, Z. (1993) · IEEE Transactions on Signal Processing · doi:10.1109/78.258082 · Cited by 9,112
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Fundamentals of statistical signal processing: Estimation theory
Prasad, K. V. (1994) · Control Engineering Practice · doi:10.1016/0967-0661(94)90195-3 · Cited by 7,902
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Pattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,459
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,388
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Greed is Good: Algorithmic Results for Sparse Approximation
Tropp, J. A. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.834793 · Cited by 3,635
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ 1 minimization
Donoho, D. L., & Elad, M. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0437847100 · Cited by 2,898
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667
A sparse signal reconstruction perspective for source localization with sensor arrays
Malioutov, D., Cetin, M., & Willsky, A. S. (2005) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2005.850882 · Cited by 2,638
Donoho, D. L. (2006) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20132 · Cited by 2,490
Applied Multivariate Statistical Analysis
Härdle, W., & Simar, L. (2003) · Springer eBooks · doi:10.1007/978-3-662-05802-2 · Cited by 2,263
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 16
Storkey, A. (2004) · neural information processing systems · Cited by 2,085
Uncertainty principles and ideal atomic decomposition
Donoho, D. L., & Huo, X. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.959265 · Cited by 1,949
Introduction to Spectral Analysis
(2005) · Technometrics · doi:10.1198/tech.2005.s841 · Cited by 1,933
Advances in Neural Information Processing Systems 18
Murray, I., Mackay, D., Zoubin Ghahramani, A., & Skilling, J. (2006) · Cited by 1,927
Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm
Gorodnitsky, I. F., & Rao, B. D. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.558475 · Cited by 1,834
Self-citation(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/15324430152748236 · Cited by 1,819
Sparse solutions to linear inverse problems with multiple measurement vectors
Cotter, S. F., Rao, B. D., Engan, K., & Kreutz-Delgado, K. (2005) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2005.849172 · Cited by 1,393
Self-citationAlgorithms for simultaneous sparse approximation. Part I: Greedy pursuit
Tropp, J. A., Gilbert, A. C., & Strauss, M. J. (2006) · Signal Processing · doi:10.1016/j.sigpro.2005.05.030 · Cited by 1,347
Better Subset Regression Using the Nonnegative Garrote
Breiman, L. (1995) · Technometrics · doi:10.1080/00401706.1995.10484371 · Cited by 1,043
Sparse representations in unions of bases
Gribonval, R., & Nielsen, M. (2003) · IEEE Transactions on Information Theory · doi:10.1109/tit.2003.820031 · Cited by 960
Algorithms for simultaneous sparse approximation. Part II: Convex relaxation
Tropp, J. A. (2006) · Signal Processing · doi:10.1016/j.sigpro.2005.05.031 · Cited by 691
A Variational Baysian Framework for Graphical Models
Attias, H. T. (1999) · neural information processing systems · Cited by 689
Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm
Gorodnitsky, I. F., George, J. S., & Rao, B. D. (1995) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(95)00107-a · Cited by 567
Self-citationOn Sparse Representations in Arbitrary Redundant Bases
Fuchs, J.-J. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.828141 · Cited by 563
Bayesian inference for variance components using only error contrasts
HARVILLE, D. A. (1974) · Biometrika · doi:10.1093/biomet/61.2.383 · Cited by 547
Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors
Moulin, P., & Liu, J. (1999) · IEEE Transactions on Information Theory · doi:10.1109/18.761332 · Cited by 500
Beal, M. J., & Ghahramani, Z. (2003) · Bayesian Statistics 7 · doi:10.1093/oso/9780198526155.003.0025 · Cited by 458
Estimation in Covariance Components Models
Dempster, A. P., Rubin, D. B., & Tsutakawa, R. K. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477653 · Cited by 452
Evaluation of gaussian processes and other methods for non-linear regression
Hinton, G. E., & Rasmussen, C. E. (1997) · Library and Archives Canada (Government of Canada) · Cited by 429
Introduction to Spectral Analysis
(2020) · Spectral Analysis for Univariate Time Series · doi:10.1017/9781139235723.002 · Cited by 413
Wavelet Shrinkage Denoising Using the Non-Negative Garrote
Gao, H.-Y. (1998) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1998.10474789 · Cited by 364
Subset selection in noise based on diversity measure minimization
Rao, B. D., Engan, K., Cotter, S. F., Palmer, J., & Kreutz-Delgado, K. (2003) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2002.808076 · Cited by 324
Self-citationAn empirical Bayesian solution to the source reconstruction problem in EEG
Phillips, C., Mattout, J., Rugg, M. D., Maquet, P., & Friston, K. J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.10.030 · Cited by 199
MEG-based imaging of focal neuronal current sources
Phillips, J. W., Leahy, R. M., & Mosher, J. C. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.585768 · Cited by 190
Variational EM algorithms for non-Gaussian latent variable models
Palmer, J. A., Kreutz-Delgado, K., Wipf, D. R., & Rao, B. D. (2006) · Cited by 170
Self-citationAdaptive Sparseness Using Jeffreys' Prior
Figueiredo, M. A. T. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0094 · Cited by 129
Reduced complexity decision feedback equalization for multipath channels with large delay spreads
Fevrier, I. J., Gelfand, S. B., & Fitz, M. P. (1999) · IEEE Transactions on Communications · doi:10.1109/26.771349 · Cited by 104
Perspectives on Sparse Bayesian Learning
Palmer, J. A., Rao, B. D., & Wipf, D. (2003) · Cited by 103
Self-citationRecovery of Jointly Sparse Signals from Few Random Projections
Wakin, M. B., Duarte, M. F., Sarvotham, S., Baron, D., & Baraniuk, R. G. (2005) · Cited by 69
Selecting Landmark Points for Sparse Manifold Learning
Silva, J. S., Marques, J. S., & Lemos, J. M. (2005) · Cited by 51
Sparse inverse solution methods for signal and image processing applications
Jeffs, B. D. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1998.681832 · Cited by 46
Basis selection in the presence of noise
Rao, B. D., & Kreutz-Delgado, K. (2002) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.1998.750962 · Cited by 20
Self-citationProbabilistic analysis for basis selection via /spl lscr//sub p/ diversity measures
Wipf, D. R., & Rao, B. D. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1326379 · Cited by 5
Self-citation
Cited by
Off-Grid Direction of Arrival Estimation Using Sparse Bayesian Inference
Yang, Z., Xie, L., & Zhang, C. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2222378 · Cited by 968
Sparse Signal Recovery With Temporally Correlated Source Vectors Using Sparse Bayesian Learning
Zhang, Z., & Rao, B. D. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2159773 · Cited by 868
Self-citationYardibi, T., Li, J., Stoica, P., Xue, M., & Baggeroer, A. B. (2010) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2010.5417172 · Cited by 699
Huang, J., & Zhang, T. (2010) · The Annals of Statistics · doi:10.1214/09-aos778 · Cited by 521
Ji, S., Dunson, D., & Carin, L. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.2005866 · Cited by 502
Metadata
- Authors
- Wipf, D. P., & Rao, B. D.
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2007-07
- Volume
- 55
- Issue
- 7
- Pages
- 3704-3716
- ISSN
- 1053-587X (ISSN-L)
- OpenAlex
- W2152279006
- Cited by
- 911
- References
- 57
- FWCI
- 16.14
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.