Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning
The DOI matches its Crossref record; 145 of the 150 references it cites were found in a registry or an index.
Abstract
This paper gives an overview of the majorization-minimization (MM) algorithmic framework, which can provide guidance in deriving problem-driven algorithms with low computational cost. A general introduction of MM is presented, including a description of the basic principle and its convergence results. The extensions, acceleration schemes, and connection to other algorithmic frameworks are also covered. To bridge the gap between theory and practice, upperbounds for a large number of basic functions, derived based on the Taylor expansion, convexity, and special inequalities, are provided as ingredients for constructing surrogate functions. With the pre-requisites established, the way of applying MM to solving specific problems is elaborated by a wide range of applications in signal processing, communications, and machine learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Convex approximation provides a natural way to design polynomial time complexity algorithms for nonconvex programs based on the principle of majorizationminimization [262] or successive convex approximation [263].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [262]
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Lee, D. D., & Seung, H. S. (2000) · neural information processing systems · Cited by 9,876
The EM Algorithm and Extensions
Kushary, D. (1998) · Technometrics · doi:10.1080/00401706.1998.10485534 · Cited by 5,417
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.
Daubechies, I., Defrise, M., & De Mol, C. (2004) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20042 · Cited by 4,958
Enhancing Sparsity by Reweighted ℓ 1 Minimization
Candès, E. J., Wakin, M. B., & Boyd, S. P. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9045-x · Cited by 4,863
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Seung, D., & Lee, L.-W. (2001) · Cited by 4,753
A practical algorithm for the determination of phase from image and diffraction plane pictures
Gerchberg, R. W. (1972) · Optik · Cited by 4,519
Iterative Solution of Nonlinear Equations in Several Variables
I., E., Ortega, J. M., & Rheinboldt, W. C. (1971) · Mathematics of Computation · doi:10.2307/2004942 · Cited by 4,408
Parikh, N., & Boyd, S. (2014) · Foundations and Trends® in Optimization · doi:10.1561/2400000003 · Cited by 4,102
On the Convergence Properties of the EM Algorithm
Wu, C. F. J. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346060 · Cited by 3,269
Sparse Principal Component Analysis
Zou, H., Hastie, T., & Tibshirani, R. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x113430 · Cited by 3,152
Matrix Differential Calculus with Applications in Statistics and Econometrics.
Magnus, J. R., & Neudecker, H. (1988) · Biometrics · doi:10.2307/2531754 · Cited by 2,699
Finding the Observed Information Matrix When Using the EM Algorithm
Louis, T. A. (1982) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1982.tb01203.x · Cited by 2,229
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 1×
“In addition, EM belongs to the class of cyclic algorithms as well [92].”
BackgroundProximal Splitting Methods in Signal Processing
Combettes, P. L., & Pesquet, J.-C. (2011) · Springer Optimization and Its Applications · doi:10.1007/978-1-4419-9569-8_10 · Cited by 2,183
Shi, Q., Razaviyayn, M., Luo, Z.-Q., & He, C. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2147784 · Cited by 2,129 · cited 1×
“This method is referred to as cyclic minimization and appears in applications such as [37], [62], [93]–[96].”
MethodConvergence of a Block Coordinate Descent Method for Nondifferentiable Minimization
Tseng, P. (2001) · Journal of Optimization Theory and Applications · doi:10.1023/a:1017501703105 · Cited by 1,989
Reconstruction of an object from the modulus of its Fourier transform
Fienup, J. R. (1978) · Optics Letters · doi:10.1364/ol.3.000027 · Cited by 1,836
Hunter, D. R., & Lange, K. (2004) · The American Statistician · doi:10.1198/0003130042836 · Cited by 1,783
Parlar, M. (2000) · Interactive Operations Research with Maple · doi:10.1007/978-1-4612-1356-7_5 · Cited by 1,741
Matrix Differential Calculus With Applications in Statistics and Econometrics
Ziegel, E. (1989) · Technometrics · doi:10.1080/00401706.1989.10488622 · Cited by 1,607
Sparse Bayesian Learning for Basis Selection
Wipf, D. P., & Rao, B. D. (2004) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2004.831016 · Cited by 1,559 · cited 2×
“Assuming Gaussianity of the observations {xi}Ni=1 , a maximum likelihood type estimator R̂ is defined as the solution of the following problem [34], [35]:”
BackgroundMethodWitten, D. M., Tibshirani, R., & Hastie, T. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp008 · Cited by 1,535
Challenges of Big Data analysis
Fan, J., Han, F., & Liu, H. (2014) · National Science Review · doi:10.1093/nsr/nwt032 · Cited by 1,482
A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization
Razaviyayn, M., Hong, M., & Luo, Z.-Q. (2013) · SIAM Journal on Optimization · doi:10.1137/120891009 · Cited by 1,304
Deterministic edge-preserving regularization in computed imaging
Charbonnier, P., Blanc-Feraud, L., Aubert, G., & Barlaud, M. (1997) · IEEE Transactions on Image Processing · doi:10.1109/83.551699 · Cited by 1,301
Iteratively reweighted least squares minimization for sparse recovery
Daubechies, I., DeVore, R., Fornasier, M., & Güntürk, C. S. (2010) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20303 · Cited by 1,295
Iterative Thresholding for Sparse Approximations
Blumensath, T., & Davies, M. E. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9035-z · Cited by 1,248
(2006) · Springer eBooks · doi:10.1007/978-3-540-35447-5 · Cited by 1,198
Constrained restoration and the recovery of discontinuities
Geman, D., & Reynolds, G. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.120331 · Cited by 1,085
Space-alternating generalized expectation-maximization algorithm
Fessler, J. A., & Hero, A. O. (1994) · IEEE Transactions on Signal Processing · doi:10.1109/78.324732 · Cited by 1,063
Phase Retrieval via Wirtinger Flow: Theory and Algorithms
Candès, E. J., Li, X., & Soltanolkotabi, M. (2015) · IEEE Transactions on Information Theory · doi:10.1109/tit.2015.2399924 · Cited by 1,049
The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data
Beaton, A. E., & Tukey, J. W. (1974) · Technometrics · doi:10.1080/00401706.1974.10489171 · Cited by 1,010
Nonlinear image recovery with half-quadratic regularization
Geman, D., & Yang, C. (1995) · IEEE Transactions on Image Processing · doi:10.1109/83.392335 · Cited by 1,000 · cited 1×
“Iteration (69) and (70) correspond to the half-quadratic minimization algorithms without over-relaxation as proposed in [7] and [8], respectively.”
BackgroundTechnical Note—A General Inner Approximation Algorithm for Nonconvex Mathematical Programs
Marks, B. R., & Wright, G. P. (1978) · Operations Research · doi:10.1287/opre.26.4.681 · Cited by 985
Power Control By Geometric Programming
Chiang, M., Tan, C. W., Palomar, D. P., O'neill, D., & Julian, D. (2007) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2007.05960 · Cited by 886
Self-citationSparse multinomial logistic regression: fast algorithms and generalization bounds
Krishnapuram, B., Carin, L., Figueiredo, M. A. T., & Hartemink, A. J. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.127 · Cited by 877
Shi, Q., Razaviyayn, M., Luo, Z.-Q., & He, C. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5946304 · Cited by 873
Optimization Transfer Using Surrogate Objective Functions
Lange, K., Hunter, D. R., & Yang, I. (2000) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2000.10474858 · Cited by 808
An, L. T. H., & Tao, P. D. (2005) · Annals of Operations Research · doi:10.1007/s10479-004-5022-1 · Cited by 800
Variations and extension of the convex–concave procedure
Lipp, T., & Boyd, S. (2016) · Optimization and Engineering · doi:10.1007/s11081-015-9294-x · Cited by 744
A Direct Formulation for Sparse PCA Using Semidefinite Programming
d'Aspremont, A., El Ghaoui, L., Jordan, M. I., & Lanckriet, G. R. G. (2007) · SIAM Review · doi:10.1137/050645506 · Cited by 678
Horst, R., & Thoai, N. V. (1999) · Journal of Optimization Theory and Applications · doi:10.1023/a:1021765131316 · Cited by 672
Algorithms for Nonnegative Matrix Factorization with the β-Divergence
Févotte, C., & Idier, J. (2011) · Neural Computation · doi:10.1162/neco_a_00168 · Cited by 663
A Distribution-Free $M$-Estimator of Multivariate Scatter
Tyler, D. E. (1987) · The Annals of Statistics · doi:10.1214/aos/1176350263 · Cited by 660
SPICE: A Sparse Covariance-Based Estimation Method for Array Processing
Stoica, P., Babu, P., & Li, J. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2090525 · Cited by 614
Self-citationDistributed weighted-multidimensional scaling for node localization in sensor networks
Costa, J. A., Patwari, N., & Hero, A. O. (2006) · ACM Transactions on Sensor Networks · doi:10.1145/1138127.1138129 · Cited by 586
Majorization–Minimization Algorithms for Wavelet-Based Image Restoration
Figueiredo, M. A. T., Bioucas-Dias, J. M., & Nowak, R. D. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.909318 · Cited by 548
Multinomial logistic regression algorithm
B�hning, D. (1992) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf00048682 · Cited by 547
Phase Retrieval Using Alternating Minimization
Netrapalli, P., Jain, P., & Sanghavi, S. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2448516 · Cited by 535
De Pierro, A. R. (1995) · IEEE Transactions on Medical Imaging · doi:10.1109/42.370409 · Cited by 475
New Algorithms for Designing Unimodular Sequences With Good Correlation Properties
Stoica, P., He, H., & Li, J. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2012562 · Cited by 475
Generalized power method for sparse principal component analysis
Journée, M., Nesterov, Y. E., Richtárik, P., & Sepulchre, R. J. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0811.4724 · Cited by 473
Maximum Likelihood Computations with Repeated Measures: Application of the EM Algorithm
Laird, N., Lange, N., & Stram, D. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478395 · Cited by 406
Complex-Valued Matrix Derivatives: With Applications in Signal Processing and Communications
Hjørungnes, A. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511921490 · Cited by 401
The Concave-Convex Procedure (CCCP)
Yuille, A., & Rangarajan, A. (2001) · Cited by 397
Optimization Methods for Designing Sequences With Low Autocorrelation Sidelobes
Song, J., Babu, P., & Palomar, D. P. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2425808 · Cited by 382
Self-citationA Gradient Algorithm Locally Equivalent to the Em Algorithm
Lange, K. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02037.x · Cited by 376
A Fast Improvement to the Em Algorithm on its Own Terms
Meilijson, I. (1989) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1989.tb01754.x · Cited by 371
Simple and Globally Convergent Methods for Accelerating the Convergence of Any EM Algorithm
VARADHAN, R., & ROLAND, C. (2008) · Scandinavian Journal of Statistics · doi:10.1111/j.1467-9469.2007.00585.x · Cited by 352
Decomposition by Partial Linearization: Parallel Optimization of Multi-Agent Systems
Scutari, G., Facchinei, F., Song, P., Palomar, D. P., & Pang, J.-S. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2293126 · Cited by 338
Self-citationGradient-based algorithms with applications to signal-recovery problems
Beck, A., & Teboulle, M. (2009) · Convex Optimization in Signal Processing and Communications · doi:10.1017/cbo9780511804458.003 · Cited by 333
Kuhn, H. W. (1973) · Mathematical Programming · doi:10.1007/bf01584648 · Cited by 329
Globally convergent algorithms for maximum a posteriori transmission tomography
Lange, K., & Fessler, J. A. (1995) · IEEE Transactions on Image Processing · doi:10.1109/83.465107 · Cited by 313
Sequence Design to Minimize the Weighted Integrated and Peak Sidelobe Levels
Song, J., Babu, P., & Palomar, D. P. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2510982 · Cited by 312
Self-citationStructured Sparse Principal Component Analysis
Jenatton, R., Obozinski, G., & Bach, F. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0909.1440 · Cited by 277
Linear Coherent Decentralized Estimation
Xiao, J.-J., Cui, S., Luo, Z.-Q., & Goldsmith, A. J. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.906762 · Cited by 277
Structured low-rank approximation and its applications
Markovsky, I. (2008) · Automatica · doi:10.1016/j.automatica.2007.09.011 · Cited by 254
Truncated Power Method for Sparse Eigenvalue Problems
Yuan, X., & Zhang, T. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.2679 · Cited by 238
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning
Mairal, J. (2015) · SIAM Journal on Optimization · doi:10.1137/140957639 · Cited by 232
Sequence Set Design With Good Correlation Properties Via Majorization-Minimization
Song, J., Babu, P., & Palomar, D. P. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2535312 · Cited by 218
Self-citationDictionary Learning for Sparse Approximations With the Majorization Method
Yaghoobi, M., Blumensath, T., & Davies, M. E. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2016257 · Cited by 213
Chouzenoux, E., Pesquet, J.-C., & Repetti, A. (2014) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-013-0465-7 · Cited by 206
Acceleration of the EM Algorithm by using Quasi-Newton Methods
Jamshidian, M., & Jennrich, R. I. (1997) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00083 · Cited by 203
Generalized Power Method for Sparse Principal Component Analysis
JournéeMichel, NesterovYurii, RichtárikPeter, & SepulchreRodolphe (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1756021 · Cited by 200
SPICE and LIKES: Two hyperparameter-free methods for sparse-parameter estimation
Stoica, P., & Babu, P. (2012) · Signal Processing · doi:10.1016/j.sigpro.2011.11.010 · Cited by 199
Self-citationConvergence of the majorization method for multidimensional scaling
de Leeuw, J. (1988) · Journal of Classification · doi:10.1007/bf01897162 · Cited by 198
A quasi-Newton acceleration for high-dimensional optimization algorithms
Zhou, H., Alexander, D., & Lange, K. (2011) · Statistics and Computing · doi:10.1007/s11222-009-9166-3 · Cited by 190
A bound optimization approach to wavelet-based image deconvolution
Figueiredo, M. A. T., & Nowak, R. D. (2005) · IEEE International Conference on Image Processing 2005 · doi:10.1109/icip.2005.1530172 · Cited by 189
Nonlinear Optimization by Successive Linear Programming
Palacios-Gomez, F., Lasdon, L., & Engquist, M. (1982) · Management Science · doi:10.1287/mnsc.28.10.1106 · Cited by 186
Total Variation-Based Image Deconvolution: a Majorization-Minimization Approach
Bioucas-Dias, J. M., Figueiredo, M. A. T., & Oliveira, J. P. (2006) · 2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings · doi:10.1109/icassp.2006.1660479 · Cited by 183
Conjugate Gradient Acceleration of the EM Algorithm
Jamshidian, M., & Jennrich, R. I. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594313 · Cited by 176
Monotonicity of quadratic-approximation algorithms
Böhning, D., & Lindsay, B. G. (1988) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf00049423 · Cited by 170
A block coordinate variable metric forward–backward algorithm
Chouzenoux, E., Pesquet, J.-C., & Repetti, A. (2016) · Journal of Global Optimization · doi:10.1007/s10898-016-0405-9 · Cited by 167
Parallel Selective Algorithms for Nonconvex Big Data Optimization
Facchinei, F., Scutari, G., & Sagratella, S. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2399858 · Cited by 162
Local Convergence of Sequential Convex Programming for Nonconvex Optimization
Dinh, Q. T., & Diehl, M. (2010) · Recent Advances in Optimization and its Applications in Engineering · doi:10.1007/978-3-642-12598-0_9 · Cited by 162
Lewitt, R. M., & Muehllehner, G. (1986) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.1986.4307734 · Cited by 155
Multiplicative Updates for Nonnegative Quadratic Programming in Support Vector Machines
Sha, F., Saul, L. K., & Lee, D. D. (2002) · ScholarlyCommons (University of Pennsylvania) · Cited by 147
Euclid in a Taxicab: Sparse Blind Deconvolution with Smoothed ${\ell _1}/{\ell _2}$ Regularization
Repetti, A., Pham, M. Q., Duval, L. C., Chouzenoux, É., & Pesquet, J.-C. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2362861 · Cited by 142
A majorization-minimization approach to the sparse generalized eigenvalue problem
Sriperumbudur, B. K., Torres, D. A., & Lanckriet, G. R. G. (2010) · Machine Learning · doi:10.1007/s10994-010-5226-3 · Cited by 138
An Expanded Theoretical Treatment of Iteration-Dependent Majorize-Minimize Algorithms
Jacobson, M. W., & Fessler, J. A. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.904387 · Cited by 138
Optimization with First-Order Surrogate FunctionsPreprint
Mairal, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.3120 · Cited by 138
A Majorize-Minimize Subspace Approach for $\ell_2-\ell_0$ Image Regularization
Chouzenoux, É., Jezierska, A., Pesquet, J.-C., & Talbot, H. (2013) · SIAM Journal on Imaging Sciences · doi:10.1137/11085997x · Cited by 129
Robust Estimation of Structured Covariance Matrix for Heavy-Tailed Elliptical Distributions
Sun, Y., Babu, P., & Palomar, D. P. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2546222 · Cited by 127
Self-citationPractical Large-Scale Optimization for Max-norm Regularization
Lee, J. D., Recht, B., Srebro, N., Tropp, J. A., & Salakhutdinov, R. (2010) · CaltechAUTHORS (California Institute of Technology) · Cited by 124
A Parallel Decomposition Method for Nonconvex Stochastic Multi-Agent Optimization Problems
Yang, Y., Scutari, G., Palomar, D. P., & Pesavento, M. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2531627 · Cited by 114
Self-citationUnified Framework to Regularized Covariance Estimation in Scaled Gaussian Models
Wiesel, A. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2170685 · Cited by 107
On global and local convergence of half-quadratic algorithms
Allain, M., Idier, J., & Goussard, Y. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.864173 · Cited by 106
Truncated power method for sparse eigenvalue problems
YuanXiao-Tong, & ZhangTong (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502610 · Cited by 104
Wu, T. T., & Lange, K. (2010) · Statistical Science · doi:10.1214/08-sts264 · Cited by 103
Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms
Ba, D. E., Babadi, B., Purdon, P. L., & Brown, E. N. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2287685 · Cited by 101
Update Rules for Parameter Estimation in Bayesian NetworksPreprint
Bauer, E. J., Koller, D., & Singer, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.1519 · Cited by 101
EM algorithms without missing data
Becker, M. P., Yang, I., & Lange, K. L. (1997) · Statistical Methods in Medical Research · doi:10.1177/096228029700600104 · Cited by 97
Multiplicative iterative algorithms for convex programming
Eggermont, P. P. B. (1990) · Linear Algebra and its Applications · doi:10.1016/0024-3795(90)90204-p · Cited by 97
Regularized Tyler's Scatter Estimator: Existence, Uniqueness, and Algorithms
Sun, Y., Babu, P., & Palomar, D. P. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2348944 · Cited by 92
Self-citationPARAMETER CONVERGENCE FOR EM AND MM ALGORITHMS
Vaida, F. (2005) · Cited by 92
A Majorize–Minimize Strategy for Subspace Optimization Applied to Image Restoration
Chouzenoux, É., Idier, J., & Moussaoui, S. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2103083 · Cited by 83
Adaptive overrelaxed bound optimization methods
Salakhutdinov, R., & Roweis, S. T. (2003) · Cited by 82
Heiser, W. J. (1995) · doi:10.1093/oso/9780198522850.003.0008 · Cited by 81
A comparison of new and old algorithms for a mixture estimation problem
Helmbold, D. P., Singer, Y., Schapire, R. E., & Warmuth, M. K. (1995) · Conference on Learning Theory · doi:10.1145/225298.225306 · Cited by 77
Bolte, J., & Pauwels, E. (2016) · Mathematics of Operations Research · doi:10.1287/moor.2015.0735 · Cited by 76
Sparse Generalized Eigenvalue Problem Via Smooth Optimization
Song, J., Babu, P., & Palomar, D. P. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2394443 · Cited by 70
Self-citationIterative Hard Thresholding and L0 Regularisation
Blumensath, T., Yaghoobi, M., & Davies, M. E. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366820 · Cited by 69
Blackbox and derivative-free optimization: theory, algorithms and applications
Audet, C., & Kokkolaras, M. (2016) · Optimization and Engineering · doi:10.1007/s11081-016-9307-4 · Cited by 67
PRIME: Phase Retrieval via Majorization-Minimization
Qiu, T., Babu, P., & Palomar, D. P. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2585084 · Cited by 67
Self-citationOğuz-Ekim, P., Gomes, J. P., Xavier, J., & Oliveira, P. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2153848 · Cited by 60
Iterative Minimization Schemes for Solving the Single Source Localization Problem
Beck, A., Teboulle, M., & Chikishev, Z. (2008) · SIAM Journal on Optimization · doi:10.1137/070698014 · Cited by 56
Majorization-minimization algorithm for smooth Itakura-Saito nonnegative matrix factorization
Févotte, C. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946898 · Cited by 55
Kullback proximal algorithms for maximum-likelihood estimation
Chrétien, S., & Hero, A. O. (2000) · IEEE Transactions on Information Theory · doi:10.1109/18.857792 · Cited by 52
Dinh, Q. T., & Diehl, M. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1107.5841 · Cited by 52
On the Convergence of the Iterative Image Space Reconstruction Algorithm for Volume ECT
(1987) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.1987.4307819 · Cited by 51
Low-Complexity Algorithms for Low Rank Clutter Parameters Estimation in Radar Systems
Sun, Y., Breloy, A., Babu, P., Palomar, D. P., Pascal, F., & Ginolhac, G. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2512535 · Cited by 50
Self-citationA Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation
Chouzenoux, É., & Pesquet, J.-C. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2709265 · Cited by 47
A Majorize-Minimize Memory Gradient method for complex-valued inverse problems
Florescu, A., Chouzenoux, É., Pesquet, J.-C., Ciuciu, P., & Ciochină, S. (2013) · Signal Processing · doi:10.1016/j.sigpro.2013.09.026 · Cited by 46
MM Algorithms for Geometric and Signomial ProgrammingPreprint
Lange, K. L., Zhou, H., & Lange, K. (2016) · Cited by 45
Radar code design for detection of moving targets
Naghsh, M. M., Soltanalian, M., Stoica, P., & Modarres‐Hashemi, M. (2014) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2014.130406 · Cited by 42
A Brief Survey of Modern Optimization for Statisticians
Lange, K., Chi, E. C., & Zhou, H. (2014) · International statistical review = Revue internationale de statistique · doi:10.1111/insr.12022 · Cited by 42
Partial Proximal Minimization Algorithms for Convex Pprogramming
Bertsekas, D. P., & Tseng, P. (1994) · SIAM Journal on Optimization · doi:10.1137/0804031 · Cited by 41
Convergence of a Sparse Representations Algorithm Applicable to Real or Complex Data
Fuchs, J. J. (2007) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2007.909363 · Cited by 40
MM algorithms for geometric and signomial programming
Lange, K., & Zhou, H. (2012) · Mathematical Programming · doi:10.1007/s10107-012-0612-1 · Cited by 38
A Comparison of New and Old Algorithms for a Mixture Estimation Problem
Helmbold, D. P., Schapire, R. E., Singer, Y., & Warmuth, M. K. (1997) · Machine Learning · doi:10.1023/a:1007301011561 · Cited by 32
Convergence of Conjugate Gradient Methods with a Closed-Form Stepsize Formula
Labat, C., & Idier, J. (2007) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-007-9306-x · Cited by 28
Robust localization in sensor networkswith iterative majorization techniques
Korkmaz, S., & van der Veen, A.-J. (2009) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2009.4960017 · Cited by 26
Flexible parallel algorithms for big data optimization
Facchinei, F., Sagratella, S., & Scutari, G. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854999 · Cited by 25
Convergence Rate Analysis of the Majorize–Minimize Subspace Algorithm
Chouzenoux, É., & Pesquet, J.-C. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2593589 · Cited by 25
Distributed Methods for Constrained Nonconvex Multi-Agent Optimization-Part I: Theory.Preprint
Scutari, G., Facchinei, F., Lampariello, L., & Song, P. (2014) · arXiv (Cornell University) · Cited by 25
Regularized Robust Estimation of Mean and Covariance Matrix Under Heavy-Tailed Distributions
Sun, Y., Babu, P., & Palomar, D. P. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2417513 · Cited by 19
Self-citationMajorize–minimize linesearch for inversion methods involving barrier function optimization
Chouzenoux, É., Moussaoui, S., & Idier, J. (2012) · Inverse Problems · doi:10.1088/0266-5611/28/6/065011 · Cited by 18
Conditions for Correct Sensor Network Localization Using SDP Relaxation
Shamsi, D., Taheri, N., Zhu, Z., & Yinyu Ye, Y. (2013) · Fields Institute communications · doi:10.1007/978-3-319-00200-2_16 · Cited by 16
Babu, P., & Stoica, P. (2012) · European Signal Processing Conference (EUSIPCO) · Cited by 11
Self-citationA Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares EstimationPreprint
Chouzenoux, É., & Pesquet, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.08722 · Cited by 6
MM Algorithms for Variance Components ModelsPreprint
Zhou, H., Hu, L., Zhou, J., & Lange, K. L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.07426 · Cited by 5
MIST: L0 Sparse Linear Regression with MomentumPreprint
Marjanovic, G., Ulfarsson, M. O., & Hero, A. O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7193 · Cited by 3
PRIME: Phase retrieval via majorization-minimization technique
Qiu, T., Babu, P., & Palomar, D. P. (2015) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2015.7421435 · Cited by 2
Self-citation
Research works printed but not linked: 7
- tipping, 2001, Sparse Bayesian learning and the relevance vector machine, J Mach Learning Res, 1, 211
- bouman, 0, Fast numerical methods for emission and transmission tomographic reconstruction, Proc Conf Inf Sci Syst, 611
- 10.1109/TSP.2015.2448516 · DOI · Paper page
- 10.2307/2290716 · DOI
- lange, 1995, A quasi-Newton acceleration of the EM algorithm, Statistica Sinica, 5, 1
- 10.1191/096228097677258219 · doi:10.1191/096228097677258219
- 10.2307/1390605 · DOI
Cited by
Multicell MIMO Communications Relying on Intelligent Reflecting Surfaces
Pan, C., Ren, H., Wang, K., Xu, W., Elkashlan, M., Nallanathan, A., & Hanzo, L. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2990766 · Cited by 1,073
Weighted Sum-Rate Maximization for Reconfigurable Intelligent Surface Aided Wireless Networks
Guo, H., Liang, Y.-C., Chen, J., & Larsson, E. G. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2970061 · Cited by 1,013
Intelligent Reflecting Surface: Practical Phase Shift Model and Beamforming Optimization
Abeywickrama, S., Zhang, R., & Yuen, C. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148961 · Cited by 901
Pan, C., Ren, H., Wang, K., Elkashlan, M., Nallanathan, A., Wang, J., & Hanzo, L. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000802 · Cited by 903
Robust and Secure Wireless Communications via Intelligent Reflecting Surfaces
Yu, X., Xu, D., Sun, Y., Ng, D. W. K., & Schober, R. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3007043 · Cited by 833
Self-citation
Metadata
- Authors
- Sun, Y., Babu, P., & Palomar, D. P.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Indian Institute of Technology Delhi (IN)
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2017-02-01
- Volume
- 65
- Issue
- 3
- Pages
- 794-816
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2508393166
- Cited by
- 1,820
- References
- 150
- Citations by year
- 2026: 200 · 2025: 294 · 2024: 300 · 2023: 248 · 2022: 230 · 2021: 201 · 2020: 135 · 2019: 110 · 2018: 77 · 2017: 19
- FWCI
- 61.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Signal Processing (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Hong Kong RGC (16207814, 16206315)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Sparse and Compressive Sensing Techniques
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Optimization and Variational Analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- majorization-minimization, surrogate functions, convex relaxation, convergence analysis, algorithm acceleration, low-complexity algorithms, signal processing, communication, machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.