The DOI matches its Crossref record; 267 of the 279 references it cites were found in a registry or an index.
Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
Open accessAbstract
Substantial progress has been made recently on developing provably accurate and efficient algorithms for low-rank matrix factorization via nonconvex optimization. While conventional wisdom often takes a dim view of nonconvex optimization algorithms due to their susceptibility to spurious local minima, simple iterative methods such as gradient descent have been remarkably successful in practice. The theoretical footings, however, had been largely lacking until recently. In this tutorial-style overview, we highlight the important role of statistical models in enabling efficient nonconvex optimization with performance guarantees. We review two contrasting approaches: (1) two-stage algorithms, which consist of a tailored initialization step followed by successive refinement; and (2) global landscape analysis and initialization-free algorithms. Several canonical matrix factorization problems are discussed, including but not limited to matrix sensing, phase retrieval, matrix completion, blind deconvolution, and robust principal component analysis. Special care is taken to illustrate the key technical insights underlying their analyses. This article serves as a testament that the integrated consideration of optimization and statistics leads to fruitful research findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 1 An example of f ( · ) taken from [21]. Here, f ( x ) = x 2 if x ∈ [ -6 , 6] and f ( x ) = x 2 +1 . 5 | x | (cos( | x | -6) -1) if | x | > 6 . It satisfies RC ( μ, λ , ζ ) with some μ, λ > 0 and ζ = ∞ , but is clearly nonconvex.](https://figures.cochat.ai/paper-figures/efaa47591b5349acbef65a17306cdcb7/nonconvex-optimization-meets-low-rank-matrix-factorization-an-overview-figure-1.png)
![Fig. 2 Illustration of the function f ( x ) = 1 4 ‖ xx ⊤ -M ‖ 2 F , where x = [ x 1 , x 2 ] ⊤ and M =[ 1 -0 . 5 -0 . 5 1 ] .](https://figures.cochat.ai/paper-figures/a0e28a4860794cd1847da664546e9c3b/nonconvex-optimization-meets-low-rank-matrix-factorization-an-overview-figure-2.png)

![Fig. 4 Comparisons of 3 designs of the preprocessing function for estimating a 64 × 64 cameraman image from phaseless measurements under Poisson noise. (Red curve) the subset scheme in (119), where the parameter γ has been optimized for each fixed α ; (Green curve) the function T ∗ α ( · ) defined in [129]; (Blue curve) the uniformly optimal design T ∗ ( · ) in [130].](https://figures.cochat.ai/paper-figures/e8c537207c1141a888de05d5b42b9ec8/nonconvex-optimization-meets-low-rank-matrix-factorization-an-overview-figure-4.png)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Fortunately, under the high-dimensional statistical setting, the local strong convexity and smoothness of the nonconvex loss functions can be exploited to tame the nonconvexity for various learning models, e.g., blind demixing [189], phase retrieval [254], and shallow neural networks [255].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [254]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–267 of 267
Gradient Descent Efficiently Finds the Cubic-Regularized Non-Convex Newton StepPreprint
Carmon, Y., & Duchi, J. C. (2016) · arXiv (Cornell University) · Cited by 50
Experimental compressive phase space tomography
Tian, L., Lee, J. W., Oh, S. B., & Barbastathis, G. (2012) · Optics Express · doi:10.1364/oe.20.008296 · Cited by 49
The Local Convexity of Solving Systems of Quadratic Equations
Sanghavi, S., Ward, R., & White, C. D. (2016) · Results in Mathematics · doi:10.1007/s00025-016-0564-5 · Cited by 49
Sparse and Low-Rank Tensor Estimation via Cubic Sketchings
Hao, B., Zhang, A. R., & Cheng, G. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.2982499 · Cited by 47
Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient DescentPreprint
Jin, C., Kakade, S. M., & Netrapalli, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08370 · Cited by 45
Efficient algorithms for collaborative filtering
Keshavan, R. H. (2012) · Cited by 41
Alternating Minimizations Converge to Second-Order Optimal Solutions
Li, Q., Zhu, Z., & Tang, G. (2019) · International Conference on Machine Learning · Cited by 41
Phase Retrieval Algorithm via Nonconvex Minimization Using a Smoothing Function
Pinilla, S., Bacca, J., & Argüello, H. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2855667 · Cited by 40
Phase Retrieval via Incremental Truncated Wirtinger FlowPreprint
Kolte, R., & Özgür, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03196 · Cited by 38
Information Recovery From Pairwise Measurements
Chen, Y., Suh, C., & Goldsmith, A. J. (2016) · IEEE Transactions on Information Theory · doi:10.1109/tit.2016.2600566 · Cited by 38
Self-citationKaczmarz Method for Solving Quadratic Equations
Chi, Y., & Lu, Y. M. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2590468 · Cited by 37
Self-citationNonconvex Demixing From Bilinear Measurements
Dong, J., & Shi, Y. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2864660 · Cited by 37
Fast, Sample-Efficient Algorithms for Structured Phase Retrieval
Jagatap, G., & Hegde, C. (2017) · neural information processing systems · Cited by 36
Blind Deconvolution by a Steepest Descent Algorithm on a Quotient Manifold
Huang, W. T., & Hand, P. (2018) · SIAM Journal on Imaging Sciences · doi:10.1137/17m1151390 · Cited by 36
Composite optimization for robust rank one bilinear sensing
Charisopoulos, V., Davis, D., Díaz, M., & Drusvyatskiy, D. (2020) · Information and Inference A Journal of the IMA · doi:10.1093/imaiai/iaaa027 · Cited by 34
Zhang, R. Y., Sojoudi, S., & Lavaei, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01631 · Cited by 33
The Global Optimization Geometry of Shallow Linear Neural Networks
Zhu, Z., Soudry, D., Eldar, Y. C., & Wakin, M. B. (2019) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-019-00889-w · Cited by 32
Complete Dictionary Recovery Using Nonconvex Optimization
Sun, J., Qu, Q., & Wright, J. N. (2015) · Cited by 31
Non-convex low-rank matrix recovery with arbitrary outliers via median-truncated gradient descent
Li, Y., Chi, Y., Zhang, H., & Liang, Y. (2019) · Information and Inference A Journal of the IMA · doi:10.1093/imaiai/iaz009 · Cited by 29
Self-citationFinding a sparse vector in a subspace: Linear sparsity using alternating directionsPreprint
Qu, Q., Sun, J., & Wright, J. N. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4659 · Cited by 29
Symmetry, Saddle Points, and Global Geometry of Nonconvex Matrix FactorizationPreprint
Li, X., Wang, Z., Lu, J., Arora, R., Haupt, J., Liu, H., & Zhao, T. (2016) · arXiv (Cornell University) · Cited by 27
Misspecified nonconvex statistical optimization for sparse phase retrieval
Yang, Z., Yang, L. F., Fang, E. X., Zhao, T., Wang, Z., & Neykov, M. (2019) · Mathematical Programming · doi:10.1007/s10107-019-01364-5 · Cited by 27
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty formPreprint
Bhojanapalli, S., Boumal, N., Jain, P., & Netrapalli, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.00186 · Cited by 26
Efficient Dictionary Learning with Gradient DescentPreprint
Gilboa, D., Buchanan, S., & Wright, J. N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10313 · Cited by 25
Global optimality in low-rank matrix optimization
Zhu, Z., Li, Q., Tang, G., & Wakin, M. B. (2017) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2017.8309166 · Cited by 25
Guarantees of Riemannian Optimization for Low Rank Matrix CompletionPreprint
Wei, K., Cai, J., Chan, T. F.-C., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06610 · Cited by 24
Spectral initialization for nonconvex estimation: High-dimensional limit and phase transitions
Lu, Y. M., & Li, G. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8007083 · Cited by 23
Self-citationA Primal-Dual Analysis of Global Optimality in Nonconvex Low-Rank Matrix Recovery
Zhang, X., Wang, L., Yu, Y., & Gu, Q. (2018) · Figshare · doi:10.60882/cispa.27231138 · Cited by 23
Complete Dictionary Learning via L4-Norm Maximization over the Orthogonal GroupPreprint
Zhai, Y., Yang, Z., Liao, Z., Wright, J. N., & Ma, Y. (2020) · arXiv (Cornell University) · Cited by 23
Stochastic Methods for Composite Optimization ProblemsPreprint
Duchi, J. C., & Ruan, F. (2017) · arXiv (Cornell University) · Cited by 21
Chen, Y., Chi, Y., Fan, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07698 · Cited by 20
Self-citationCoordinate descent algorithms for phase retrieval
Zeng, W., & So, H. C. (2019) · Signal Processing · doi:10.1016/j.sigpro.2019.107418 · Cited by 20
On critical points of quadratic low-rank matrix optimization problems
Uschmajew, A., & Vandereycken, B. (2020) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/drz061 · Cited by 20
Hong, M., Razaviyayn, M., & Lee, J. D. (2018) · Cited by 19
Solving Systems of Phaseless Equations via Riemannian Optimization with Optimal Sampling Complexity
Cai, J., & Wei, K. (2023) · Journal of Computational Mathematics · doi:10.4208/jcm.2207-m2021-0247 · Cited by 19
Phase retrieval using iterative projections: Dynamics in the large systems limit
Li, G., Gu, Y., & Lu, Y. M. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447133 · Cited by 18
Self-citationA non-convex blind calibration method for randomised sensing strategies
Cambareri, V., & Jacques, L. (2016) · International Workshop on Compressed Sensing Theory and its Applications to Radar, Sonar and Remote Sensing · doi:10.1109/cosera.2016.7745690 · Cited by 18
Information recovery from pairwise measurements
Chen, Y., & Goldsmith, A. J. (2014) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2014.6875186 · Cited by 18
Self-citationPhase retrieval using alternating minimization in a batch setting
Zhang, T. (2019) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2019.02.001 · Cited by 18
Exploiting the Structure Effectively and Efficiently in Low-Rank Matrix Recovery
Cai, J., & Wei, K. (2018) · Handbook of numerical analysis · doi:10.1016/bs.hna.2018.09.001 · Cited by 18
Composite optimization for robust blind deconvolutionPreprint
Charisopoulos, V., Davis, D., Díaz, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01624 · Cited by 17
Stochastic Methods for Composite and Weakly Convex Optimization ProblemsPreprint
Duchi, J. C., & Ruan, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.08570 · Cited by 16
Complete Dictionary Learning via $\ell^4$-Norm Maximization over the Orthogonal GroupPreprint
Zhai, Y., Yang, Z., Liao, Z., Wright, J. N., & Ma, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02435 · Cited by 15
How Much Restricted Isometry is Needed In Nonconvex Matrix Recovery?Preprint
Zhang, R. Y., Josz, C., Sojoudi, S., & Lavaei, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10251 · Cited by 15
Solving systems of phaseless equations via Riemannian optimization with optimal sampling complexityPreprint
Cai, J., & Wei, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02773 · Cited by 13
When Are Nonconvex Optimization Problems Not Scary?
Sun, J. (2016) · doi:10.7916/d8251j7h · Cited by 13
Local Geometry of One-Hidden-Layer Neural Networks for Logistic Regression.Preprint
Fu, H., Chi, Y., & Liang, Y. (2018) · arXiv (Cornell University) · Cited by 12
Self-citationCoherence Retrieval Using Trace Regularization
Bao, C., Barbastathis, G., Ji, H. Y., Shen, Z., & Zhang, Z. (2018) · SIAM Journal on Imaging Sciences · doi:10.1137/17m1124097 · Cited by 12
Low-rank regularization and solution uniqueness in over-parameterized matrix sensing.
Geyer, K., Kyrillidis, A., & Kalev, A. (2020) · International Conference on Artificial Intelligence and Statistics · Cited by 11
Blind gain and phase calibration for low-dimensional or sparse signal sensing via power iteration
Li, Y., Lee, K., & Bresler, Y. (2017) · International Conference on Sampling Theory and Applications · doi:10.1109/sampta.2017.8024422 · Cited by 10
Leave-one-out Approach for Matrix Completion: Primal and Dual AnalysisPreprint
Ding, L., & Chen, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07554 · Cited by 9
Self-citationChen, J., Liu, D., & Li, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.06116 · Cited by 9
Ji, C., & Li, X. (2017) · arXiv (Cornell University) · Cited by 9
Coordinate Descent Algorithms for Phase RetrievalPreprint
Zeng, W., & So, H. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.03474 · Cited by 8
The Geometry of Equality-constrained Global Consensus Problems
Li, Q., Zhu, Z., Tang, G., & Wakin, M. B. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682568 · Cited by 8
Nonconvex Low-Rank Symmetric Tensor Completion from Noisy DataPreprint
Cai, C., Li, G., Poor, H. V., & Chen, Y. (2019) · arXiv (Cornell University) · Cited by 8
Self-citationConvex and nonconvex geometries of symmetric tensor factorization
Li, Q., & Tang, G. (2017) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2017.8335189 · Cited by 8
Approximate Message Passing for Amplitude Based OptimizationPreprint
Ma, J., Xu, J., & Maleki, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03276 · Cited by 8
PCA by Determinant Optimisation has no Spurious Local Optima
Hauser, R., Eftekhari, A., & Matzinger, H. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220069 · Cited by 6
Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient DescentPreprint
Li, Y., Chi, Y., Zhang, H., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.08114 · Cited by 5
Self-citationPhase Retrieval Using Alternating Minimization in a Batch Setting
Zhang, T. (2018) · Information Theory and Applications Workshop (ITA) · doi:10.1109/ita.2018.8503171 · Cited by 5
The Global Optimization Geometry of Shallow Linear Neural NetworksPreprint
Zhu, Z., Soudry, D., Eldar, Y. C., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04938 · Cited by 4
Solving Quadratic Equations via Amplitude-based Nonconvex Optimization
Monardo, V., Li, Y., & Chi, Y. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682357 · Cited by 2
Self-citationImplicit regularization and solution uniqueness in over-parameterized matrix sensingPreprint
Geyer, K., Kyrillidis, A., & Kalev, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02046 · Cited by 1
Exploiting the structure effectively and efficiently in low rank matrix recoveryPreprint
Cai, J., & Wei, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03652 · Cited by 1
Fast blind deconvolution and blind demixing via nonconvex optimization
Ling, S., & Strohmer, T. (2017) · International Conference on Sampling Theory and Applications · doi:10.1109/sampta.2017.8024387 · Cited by 1
Gradient Descent Finds the Cubic-Regularized Non-Convex Newton StepPreprint
Carmon, Y., & Duchi, J. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00547
Research works printed but not linked: 5
- allen-zhu, 0, Proc Neural Inf Process Syst, 3716
- qu, 0, Convolutional phase retrieval, Proc Neural Inf Process Syst, 6086
- allen-zhu, 2017, Recent advances in stochastic convex and non-convex optimization, ICML Tut
- duchi, 0, Efficient projections onto the $\ell _1$-ball for learning in high dimensions, Proc Int Conf Mach Learn, 272
- zhao, 0, Proc Adv Neural Inf Process Syst, 559
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“, blind demixing [189], phase retrieval [254], and shallow neural networks [255].”
BackgroundExploiting Shared Representations for Personalized Federated LearningPreprint
Collins, L., Hassani, H., Mokhtari, A., & Shakkottai, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07078 · Cited by 1,317
Optimization for deep learning: theory and algorithmsPreprint
Sun, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.08957 · Cited by 283
Ma, C., Wang, K., Chi, Y., & Chen, Y. (2020) · Foundations of Computational Mathematics · doi:10.1007/s10208-019-09429-9 · Cited by 204
Self-citationGradient descent with random initialization: fast global convergence for nonconvex phase retrieval
Chen, Y., Chi, Y., Fan, J., & Ma, C. (2019) · Mathematical Programming · doi:10.1007/s10107-019-01363-6 · Cited by 224
Self-citation
Metadata
- Authors
- Chi, Y., Lu, Y. M., & Chen, Y.
- Affiliations
- Carnegie Mellon University (US)
- Harvard University (US)
- Princeton University (US)
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2019-10-15
- Volume
- 67
- Issue
- 20
- Pages
- 5239-5269
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2969215180
- Cited by
- 360
- References
- 279
- Citations by year
- 2026: 20 · 2025: 33 · 2024: 36 · 2023: 50 · 2022: 40 · 2021: 80 · 2020: 65 · 2019: 32
- FWCI
- 37.03
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- IEEE Transactions on Signal Processing (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Office of Naval Research (N00014-18-1-2142, N00014-19-1-2404)
- Army Research Office (W911NF-18-1-0303)
- Air Force Office of Scientific Research (FA9550-15-1-0205)
- National Science Foundation (CCF-1901199, CCF-1806154, ECCS-1818571, CCF-1718698, CCF-1910410, CCF-1907661, IIS-1900140)
- Harvard Dean's Competitive Fund for Promising Research
- AFOSR (YIP FA9550-19-1-0030)
- ARO (W911NF-18-1-0303)
- ONR (N00014-19-1-2120)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Sparse and Compressive Sensing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Mathematics › Numerical Analysis › Advanced Optimization Algorithms Research
- Keywords
- non-convex optimization, low-rank factorization, statistical models, spurious local minima, two-stage algorithm, matrix sensing, phase retrieval, matrix completion, blind deconvolution, robust principal component analysis, performance guarantees
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.