Proximal Algorithms
The DOI matches its Crossref record; 180 of the 182 references it cites were found in a registry or an index.
Abstract
This monograph is about a class of optimization algorithms called proximal algorithms. Much like Newton’s method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, large-scale, or distributed versions of these problems. They are very generally applicable, but are especially well-suited to problems of substantial recent interest involving large or high-dimensional datasets. Proximal methods sit at a higher level of abstraction than classical algorithms like Newton’s method: the base operation is evaluating the proximal operator of a function, which itself involves solving a small convex optimization problem. These subproblems, which generalize the problem of projecting a point onto a convex set, often admit closed-form solutions or can be solved very quickly with standard or simple specialized methods. Here, we discuss the many different interpretations of proximal operators and algorithms, describe their connections to many other topics in optimization and applied mathematics, survey some popular algorithms, and provide a large number of examples of proximal operators that commonly arise in practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Self-citationRegression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Self-citation(2021) · A First Course in Statistical Programming with R · doi:10.1017/9781108993456.011 · Cited by 13,724
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Self-citationA method for the solution of certain non-linear problems in least squares
Levenberg, K. (1944) · Quarterly of Applied Mathematics · doi:10.1090/qam/10666 · Cited by 12,359
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
De-noising by soft-thresholding
Donoho, D. L. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382009 · Cited by 9,619
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261
Robust estimation of a location parameter
Huber, P. J. (1964) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177703732 · Cited by 7,228
A Singular Value Thresholding Algorithm for Matrix Completion
Cai, J.-F., Candès, E. J., & Shen, Z. (2010) · SIAM Journal on Optimization · doi:10.1137/080738970 · Cited by 5,809
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (2001) · SIAM Review · doi:10.1137/s003614450037906x · Cited by 5,085
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.
Daubechies, I., Defrise, M., & De Mol, C. (2004) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20042 · Cited by 4,958
A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging
Chambolle, A., & Pock, T. (2011) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-010-0251-1 · Cited by 4,336
Constrained Optimization and Lagrange Multiplier Methods
(1982) · Elsevier eBooks · doi:10.1016/c2013-0-10366-2 · Cited by 4,016
Partitioning procedures for solving mixed-variables programming problems
Benders, J. F. (1962) · Numerische Mathematik · doi:10.1007/bf01386316 · Cited by 3,855
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719 · cited 1×
“Some classic and important modern references include those by Dantzig and Wolfe [66], Benders [23], Lasdon [118], Geoffrion [93], Tsitsiklis [189], Bertsekas and Tsitsklis [27], Censor and Zenios [50], and Nedic̀ and Ozdaglar [144, 145].”
BackgroundMonotone Operators and the Proximal Point Algorithm
Rockafellar, R. T. (1976) · SIAM Journal on Control and Optimization · doi:10.1137/0314056 · Cited by 3,559
Introductory Lectures on Convex Optimization: A Basic Course
Nesterov, Y. E. (2014) · Medical Entomology and Zoology · Cited by 3,557
Finite-Dimensional Variational Inequalities and Complementarity Problems
(2004) · Springer Series in Operations Research and Financial Engineering · doi:10.1007/b97544 · Cited by 3,521
Convex Analysis and Monotone Operator Theory in Hilbert Spaces
Bauschke, H. H., & Combettes, P. L. (2017) · CMS Books in Mathematics · doi:10.1007/978-3-319-48311-5 · Cited by 3,307
Existence of an equilibrium for a competitive economy
Arrow, K. J., Debreu, G., Debreu, G., & Hildenbrand, W. (2008) · Mathematical Economics · doi:10.1017/ccol052123736x.005 · Cited by 3,041
Two-Point Step Size Gradient Methods
BARZILAI, J., & BORWEIN, J. M. (1988) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/8.1.141 · Cited by 2,873
Signal Recovery by Proximal Forward-Backward Splitting
Combettes, P. L., & Wajs, V. R. (2005) · Multiscale Modeling & Simulation · doi:10.1137/050626090 · Cited by 2,678
Bregman, L. M. (1967) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(67)90040-7 · Cited by 2,618
Operateurs Maximaux Monotones - Et Semi-Groupes De Contractions Dans Les Espaces De Hilbert
(1973) · North-Holland Mathematics Studies · doi:10.1016/s0304-0208(08)x7125-7 · Cited by 2,578
High dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · Cited by 2,539
High-dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · The Annals of Statistics · doi:10.1214/009053606000000281 · Cited by 2,521 · cited 1×
“[201], sum-of-norms regularization [156], the CAP family of penalties [206], l1 trend filtering [114], covariance selection [71], sparse recovery [43], basis pursuit [56], Huber loss [105], singular value thresholding [45], and sparse graph selection [138].”
BackgroundA dual algorithm for the solution of nonlinear variational problems via finite element approximation
Gabay, D., & Mercier, B. (1976) · Computers & Mathematics with Applications · doi:10.1016/0898-1221(76)90003-1 · Cited by 2,514
Mean value methods in iteration
Mann, W. R. (1953) · Proceedings of the American Mathematical Society · doi:10.1090/s0002-9939-1953-0054846-3 · Cited by 2,505
Generalized Benders decomposition
Geoffrion, A. M. (1972) · Journal of Optimization Theory and Applications · doi:10.1007/bf00934810 · Cited by 2,478
Smooth minimization of non-smooth functions
Nesterov, Y. (2005) · Mathematical Programming · doi:10.1007/s10107-004-0552-5 · Cited by 2,455
Decomposition Principle for Linear Programs
Dantzig, G. B., & Wolfe, P. (1960) · Operations Research · doi:10.1287/opre.8.1.101 · Cited by 2,315
Proximal Splitting Methods in Signal Processing
Combettes, P. L., & Pesquet, J.-C. (2011) · Springer Optimization and Its Applications · doi:10.1007/978-1-4419-9569-8_10 · Cited by 2,183
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
Bruckstein, A. M., Donoho, D. L., & Elad, M. (2009) · SIAM Review · doi:10.1137/060657704 · Cited by 2,183
A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration
Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.909319 · Cited by 2,064 · cited 1×
“Finally, there are ways to accelerate the basic proximal gradient method other than the method we showed, such as through the use of Barzilai-Borwein step sizes [6, 199] or with other types of extrapolation steps [28].”
MethodStoer, J. (1999) · Springer-Lehrbuch · doi:10.1007/978-3-662-09021-3 · Cited by 2,059
Splitting Algorithms for the Sum of Two Nonlinear Operators
Lions, P. L., & Mercier, B. (1979) · SIAM Journal on Numerical Analysis · doi:10.1137/0716071 · Cited by 1,972
Sparse Reconstruction by Separable Approximation
Wright, S. J., Nowak, R. D., & Figueiredo, M. A. T. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2016892 · Cited by 1,854
Harker, P. T., & Pang, J.-S. (1990) · Mathematical Programming · doi:10.1007/bf01582255 · Cited by 1,817
Hunter, D. R., & Lange, K. (2004) · The American Statistician · doi:10.1198/0003130042836 · Cited by 1,783
(1989) · Choice Reviews Online · doi:10.5860/choice.26-5109 · Cited by 1,755
On the numerical solution of heat conduction problems in two and three space variables
Douglas, J., & Rachford, H. H. (1956) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1956-0084194-4 · Cited by 1,744
Optimization Theory of Large Systems
Lasdon, L. S., & Tabak, D. (1971) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1971.4308301 · Cited by 1,677
On Projection Algorithms for Solving Convex Feasibility Problems
Bauschke, H. H., & Borwein, J. M. (1996) · SIAM Review · doi:10.1137/s0036144593251710 · Cited by 1,654
Proximité et dualité dans un espace hilbertien
Moreau, J. J. (1965) · Bulletin de la Société mathématique de France · doi:10.24033/bsmf.1625 · Cited by 1,462
Problem complexity and method efficiency in optimization
(1984) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(84)90101-0 · Cited by 1,329
PROBLEMS IN DECENTRALIZED DECISION MAKING AND COMPUTATION
Τσιτσικλής, Ι. (2014) · doi:10.12681/eadd/3778 · Cited by 1,278
A Modified Forward-Backward Splitting Method for Maximal Monotone Mappings
Tseng, P. (2000) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012998338806 · Cited by 1,223
An EM algorithm for wavelet-based image restoration
Figueiredo, M. A. T., & Nowak, R. D. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.814255 · Cited by 1,219
Efficient projections onto the l1-ball for learning in high dimensions
Duchi, J., Shalev-Shwartz, S., Singer, Y., & Chandra, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390191 · Cited by 1,184
Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming
Rockafellar, R. T. (1976) · Mathematics of Operations Research · doi:10.1287/moor.1.2.97 · Cited by 1,175
Monotone (nonlinear) operators in Hilbert space
Minty, G. J. (1962) · Duke Mathematical Journal · doi:10.1215/s0012-7094-62-02933-2 · Cited by 1,135
Glowinski, R., & Marroco, A. (1975) · Revue française d'automatique, informatique, recherche opérationnelle. Analyse numérique · doi:10.1051/m2an/197509r200411 · Cited by 1,105
Mirror descent and nonlinear projected subgradient methods for convex optimization
Beck, A., & Teboulle, M. (2003) · Operations Research Letters · doi:10.1016/s0167-6377(02)00231-6 · Cited by 1,060
On the maximality of sums of nonlinear monotone operators
Rockafellar, R. T. (1970) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1970-0282272-5 · Cited by 949
Toh, K.-C., & Yun, S. (2009) · National University of Singapore · Cited by 923
Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and Convergence
Hale, E. T., Yin, W., & Zhang, Y. (2008) · SIAM Journal on Optimization · doi:10.1137/070698920 · Cited by 909
NESTA: A Fast and Accurate First-Order Method for Sparse Recovery
Becker, S., Bobin, J., & Candès, E. J. (2009) · Cited by 897
On the $O(1/n)$ Convergence Rate of the Douglas–Rachford Alternating Direction Method
He, B., & Yuan, X. (2012) · SIAM Journal on Numerical Analysis · doi:10.1137/110836936 · Cited by 881
Gradient methods for minimizing composite objective function
Nesterov, Y. E. (2007) · Cited by 867
Parallel optimization: Theory, algorithms, and applications
(1998) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(98)90713-1 · Cited by 813
CVXGEN: a code generator for embedded convex optimization
Mattingley, J., & Boyd, S. (2012) · Optimization and Engineering · doi:10.1007/s11081-011-9176-9 · Cited by 790
Self-citationEvaluation Measures for Multi-class Subgroup Discovery
Abudawood, T., & Flach, P. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-04180-8_20 · Cited by 778
On the maximal monotonicity of subdifferential mappings
Rockafellar, R. (1970) · Pacific Journal of Mathematics · doi:10.2140/pjm.1970.33.209 · Cited by 740
Esser, E., Zhang, X., & Chan, T. F. (2010) · SIAM Journal on Imaging Sciences · doi:10.1137/09076934x · Cited by 695
Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction
Zhang, X., Burger, M., Bresson, X., & Osher, S. (2010) · SIAM Journal on Imaging Sciences · doi:10.1137/090746379 · Cited by 688
Rank-Sparsity Incoherence for Matrix Decomposition
Chandrasekaran, V., Sanghavi, S., Parrilo, P. A., & Willsky, A. S. (2011) · SIAM Journal on Optimization · doi:10.1137/090761793 · Cited by 686
On the Convergence of the Proximal Point Algorithm for Convex Minimization
Güler, O. (1991) · SIAM Journal on Control and Optimization · doi:10.1137/0329022 · Cited by 681
Adaptive Restart for Accelerated Gradient Schemes
O’Donoghue, B., & Candès, E. (2015) · Foundations of Computational Mathematics · doi:10.1007/s10208-013-9150-3 · Cited by 642
Risk-sensitive optimal control
Whittle, P. J. L. (1990) · Medical Entomology and Zoology · Cited by 641
The composite absolute penalties family for grouped and hierarchical variable selection
Zhao, P., Rocha, G., & Yu, B. (2009) · The Annals of Statistics · doi:10.1214/07-aos584 · Cited by 631
Ergodic convergence to a zero of the sum of monotone operators in Hilbert space
Passty, G. B. (1979) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(79)90234-8 · Cited by 609
Optimization with Sparsity-Inducing Penalties
Bach, F. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985118 · Cited by 608
Templates for convex cone problems with applications to sparse signal recovery
Becker, S. R., Candès, E. J., & Grant, M. C. (2011) · Mathematical Programming Computation · doi:10.1007/s12532-011-0029-5 · Cited by 608
An Algorithm for Restricted Least Squares Regression
Dykstra, R. L. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477029 · Cited by 551
Majorization–Minimization Algorithms for Wavelet-Based Image Restoration
Figueiredo, M. A. T., Bioucas-Dias, J. M., & Nowak, R. D. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.909318 · Cited by 548
Fonctions convexes duales et points proximaux dans un espace hilbertien
Moreau, J. J. (1962) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 501
A Douglas–Rachford Splitting Approach to Nonsmooth Convex Variational Signal Recovery
Combettes, P. L., & Pesquet, J.-C. (2007) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2007.910264 · Cited by 490
Tseng, P. (1991) · SIAM Journal on Control and Optimization · doi:10.1137/0329006 · Cited by 467
Solving monotone inclusions via compositions of nonexpansive averaged operators
Combettes *, P. L. (2004) · Optimization · doi:10.1080/02331930412331327157 · Cited by 457
A dual approach to solving nonlinear programming problems by unconstrained optimization
Rockafellar, R. T. (1973) · Mathematical Programming · doi:10.1007/bf01580138 · Cited by 443
Brève communication. Régularisation d'inéquations variationnelles par approximations successives
Martinet, B. (1970) · Revue française d'informatique et de recherche opérationnelle. Série rouge · doi:10.1051/m2an/197004r301541 · Cited by 422
A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration
Zhang, X., Burger, M., & Osher, S. (2011) · Journal of Scientific Computing · doi:10.1007/s10915-010-9408-8 · Cited by 419
Latent variable graphical model selection via convex optimization
Chandrasekaran, V., Parrilo, P. A., & Willsky, A. S. (2012) · The Annals of Statistics · doi:10.1214/11-aos949 · Cited by 377
Alternating direction augmented Lagrangian methods for semidefinite programming
Wen, Z., Goldfarb, D., & Yin, W. (2010) · Mathematical Programming Computation · doi:10.1007/s12532-010-0017-1 · Cited by 359
Bertsekas, D. P. (1976) · Automatica · doi:10.1016/0005-1098(76)90077-7 · Cited by 347
Convergence theorems for sequences of nonlinear operators in Banach spaces
Browder, F. E. (1967) · Mathematische Zeitschrift · doi:10.1007/bf01109805 · Cited by 341
Prox-regular functions in variational analysis
Poliquin, R. A., & Rockafellar, R. T. (1996) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-96-01544-9 · Cited by 333
Gradient-based algorithms with applications to signal-recovery problems
Beck, A., & Teboulle, M. (2009) · Convex Optimization in Signal Processing and Communications · doi:10.1017/cbo9780511804458.003 · Cited by 333
Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming
Eckstein, J. (1993) · Mathematics of Operations Research · doi:10.1287/moor.18.1.202 · Cited by 324
Convergence Rates in Forward--Backward Splitting
Chen, G. H.-G., & Rockafellar, R. T. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623495290179 · Cited by 285
Proximal Methods for Hierarchical Sparse Coding
Jenatton, R., Mairal, J., Obozinski, G., & Bach, F. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.2139 · Cited by 272
Proximal minimization algorithm withD-functions
Censor, Y., & Zenios, S. A. (1992) · Journal of Optimization Theory and Applications · doi:10.1007/bf00940051 · Cited by 268
Smoothing and First Order Methods: A Unified Framework
Beck, A., & Teboulle, M. (2012) · SIAM Journal on Optimization · doi:10.1137/100818327 · Cited by 267
Linear convergence of iterative soft-thresholdingPreprint
Bredies, K., & Lorenz, D. A. (2012) · Cited by 263
Nonlinear maximal monotone operators in Banach space
Browder, F. E. (1968) · Mathematische Annalen · doi:10.1007/bf01418765 · Cited by 261
Splitting methods for monotone operators with applications to parallel optimization
Eckstein, J. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 250
Unbiased Risk Estimates for Singular Value Thresholding and Spectral Estimators
Candes, E. J., Sing-Long, C. A., & Trzasko, J. D. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2270464 · Cited by 243
Entropic Proximal Mappings with Applications to Nonlinear Programming
Teboulle, M. (1992) · Mathematics of Operations Research · doi:10.1287/moor.17.3.670 · Cited by 240
Nonlinear monotone operators and convex sets in Banach spaces
Browder, F. E. (1965) · Bulletin of the American Mathematical Society · doi:10.1090/s0002-9904-1965-11391-x · Cited by 239
Practical Aspects of the Moreau--Yosida Regularization: Theoretical Preliminaries
Lemaréchal, C., & Sagastizábal, C. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623494267127 · Cited by 236
New Proximal Point Algorithms for Convex Minimization
Güler, O. (1992) · SIAM Journal on Optimization · doi:10.1137/0802032 · Cited by 231
Alternating Projections on Manifolds
Lewis, A. S., & Malick, J. (2008) · Mathematics of Operations Research · doi:10.1287/moor.1070.0291 · Cited by 221
Set-Valued Mappings and Enlargements of Monotone Operators
(2008) · Optimization and Its Applications · doi:10.1007/978-0-387-69757-4 · Cited by 213
On the monotonicity of the gradient of a convex function
Minty, G. (1964) · Pacific Journal of Mathematics · doi:10.2140/pjm.1964.14.243 · Cited by 205
Proximal Thresholding Algorithm for Minimization over Orthonormal Bases
Combettes, P. L., & Pesquet, J.-C. (2008) · SIAM Journal on Optimization · doi:10.1137/060669498 · Cited by 190
A bound optimization approach to wavelet-based image deconvolution
Figueiredo, M. A. T., & Nowak, R. D. (2005) · IEEE International Conference on Image Processing 2005 · doi:10.1109/icip.2005.1530172 · Cited by 189
Convex Analysis on the Hermitian Matrices
Lewis, A. S. (1996) · SIAM Journal on Optimization · doi:10.1137/0806009 · Cited by 187
Parallel Algorithms for Dense Linear Algebra Computations
Gallivan, K. A., Plemmons, R. J., & Sameh, A. H. (1990) · SIAM Review · doi:10.1137/1032002 · Cited by 186
Cooperative distributed multi-agent optimization
Nedić, A., & Ozdaglar, A. (2009) · Convex Optimization in Signal Processing and Communications · doi:10.1017/cbo9780511804458.011 · Cited by 185
An efficient projection for l 1 , ∞ regularization
Quattoni, A., Carreras, X., Collins, M., & Darrell, T. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553484 · Cited by 184
Dykstra′s Alternating Projection Algorithm for Two Sets
Bauschke, H. H., & Borwein, J. M. (1994) · Journal of Approximation Theory · doi:10.1006/jath.1994.1136 · Cited by 181
Iterative Refinement in Floating Point
Moler, C. B. (1967) · Journal of the ACM · doi:10.1145/321386.321394 · Cited by 176
A Splitting Method for Optimal Control
O'Donoghue, B., Stathopoulos, G., & Boyd, S. (2013) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2012.2231960 · Cited by 172
Self-citationThe Convex Analysis of Unitarily Invariant Matrix Functions
Lewis, A. S. (1995) · Journal of convex analysis · doi:10.68381/jca02012 · Cited by 171
A generalized proximal point algorithm for certain non-convex minimization problems
FUKUSHIMA, M., & MINE, H. (1981) · International Journal of Systems Science · doi:10.1080/00207728108963798 · Cited by 166
Asymptotic Convergence Analysis of the Proximal Point Algorithm
Luque, F. J. (1984) · SIAM Journal on Control and Optimization · doi:10.1137/0322019 · Cited by 164
Segmentation of ARX-models using sum-of-norms regularization
Ohlsson, H., Ljung, L., & Boyd, S. (2010) · Automatica · doi:10.1016/j.automatica.2010.03.013 · Cited by 158
Self-citationParallel Proximal Algorithm for Image Restoration Using Hybrid Regularization
Pustelnik, N., Chaux, C., & Pesquet, J.-C. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2128335 · Cited by 154
Derivatives of Spectral Functions
Lewis, A. S. (1996) · Mathematics of Operations Research · doi:10.1287/moor.21.3.576 · Cited by 154
Walras' Tatonnement in the Theory of Exchange
Uzawa, H. (1960) · The Review of Economic Studies · doi:10.2307/2296080 · Cited by 149
Tseng, P. (1990) · Mathematical Programming · doi:10.1007/bf01582258 · Cited by 135
Finite termination of the proximal point algorithm
Ferris, M. C. (1991) · Mathematical Programming · doi:10.1007/bf01594944 · Cited by 135
Block splitting for distributed optimization
Parikh, N., & Boyd, S. (2013) · Mathematical Programming Computation · doi:10.1007/s12532-013-0061-8 · Cited by 132
Self-citationPractical Large-Scale Optimization for Max-norm Regularization
Lee, J. D., Recht, B., Srebro, N., Tropp, J. A., & Salakhutdinov, R. (2010) · CaltechAUTHORS (California Institute of Technology) · Cited by 124
Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities
Tseng, P. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623495279797 · Cited by 123
Statistical estimation and testing via the sorted L1 normPreprint
Bogdan, M., van den Berg, E., Su, W., & Candès, E. J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.1969 · Cited by 118
Sparse Inverse Covariance Selection via Alternating Linearization MethodsPreprint
Scheinberg, K., Ma, S., & Goldfarb, D. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1011.0097 · Cited by 117
Epigraphical projection and proximal tools for solving constrained convex optimization problems
Chierchia, G., Pustelnik, N., Pesquet, J.-C., & Pesquet‐Popescu, B. (2014) · Signal Image and Video Processing · doi:10.1007/s11760-014-0664-1 · Cited by 117
All convex invariant functions of hermitian matrices
Davis, C. (1957) · Archiv der Mathematik · doi:10.1007/bf01898787 · Cited by 116
Note on the iterative refinement of least squares solution
Golub, G. H., & Wilkinson, J. H. (1966) · Numerische Mathematik · doi:10.1007/bf02166032 · Cited by 112
Proximal Point Methods and Nonconvex Optimization
Kaplan, A., & Tichatschke, R. (1998) · Journal of Global Optimization · doi:10.1023/a:1008321423879 · Cited by 108
A family of variable metric proximal methods
Bonnans, J. F., Gilbert, J. C., Lemaréchal, C., & Sagastizábal, C. A. (1995) · Mathematical Programming · doi:10.1007/bf01585756 · Cited by 108
A quasi-Newton proximal splitting methodPreprint
Becker, S., & Fadili, J. M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.1156 · Cited by 107
Friedland, S. (1981) · Linear and Multilinear Algebra · doi:10.1080/03081088108817381 · Cited by 106
The Proximal Average: Basic Theory
Bauschke, H. H., Goebel, R. K., Lucet, Y., & Wang, X. (2008) · SIAM Journal on Optimization · doi:10.1137/070687542 · Cited by 104
Fast First-Order Methods for Composite Convex Optimization with Backtracking
Scheinberg, K., Goldfarb, D., & Bai, X. (2014) · Foundations of Computational Mathematics · doi:10.1007/s10208-014-9189-9 · Cited by 102
Multi-valued monotone nonlinear mappings and duality mappings in Banach spaces
Browder, F. E. (1965) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1965-0180884-9 · Cited by 100
Sparse and Low-Rank Matrix Decompositions
Chandrasekaran, V., Sanghavi, S., Parrilo, P. A., & Willsky, A. S. (2009) · IFAC Proceedings Volumes · doi:10.3182/20090706-3-fr-2004.00249 · Cited by 100
Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection
Ma, S., Xue, L., & Zou, H. (2013) · Neural Computation · doi:10.1162/neco_a_00379 · Cited by 99
Projection algorithms and monotone operators
Borwein, J. M., & Bauschke, H. H. (1996) · Cited by 95
A Globally and Superlinearly Convergent Algorithm for Nonsmooth Convex Minimization
Fukushima, M., & Qi, L. (1996) · SIAM Journal on Optimization · doi:10.1137/s1052623494278839 · Cited by 95
An iterative solution of a variational inequality for certain monotone operators in Hilbert space
Bruck, R. E. (1975) · Bulletin of the American Mathematical Society · doi:10.1090/s0002-9904-1975-13874-2 · Cited by 94
Bauschke, H. H., & Koch, V. (2015) · Contemporary mathematics - American Mathematical Society · doi:10.1090/conm/636/12726 · Cited by 85
Nonlinear rescaling and proximal-like methods in convex optimization
Polyak, R. A., & Teboulle, M. (1997) · Mathematical Programming · doi:10.1007/bf02614440 · Cited by 85
Message-passing for Graph-structured Linear Programs: Proximal Methods and Rounding Schemes
Ravikumar, P., Agarwal, A., & Wainwright, M. J. (2010) · doi:10.5555/1756006.1756040 · Cited by 83
Projections onto order simplexes
Grotzinger, S. J., & Witzgall, C. (1984) · Applied Mathematics & Optimization · doi:10.1007/bf01449044 · Cited by 77
NON-LINEAR MONOTONE OPERATORS IN BANACH SPACES
Kachurovskii, R. I. (1968) · Russian Mathematical Surveys · doi:10.1070/rm1968v023n02abeh001239 · Cited by 67
Coupling optimization methods and variational convergence
Lemaire, B. (1988) · Birkhäuser Basel eBooks · doi:10.1007/978-3-0348-9297-1_12 · Cited by 62
Performance Bounds and Suboptimal Policies for Multi-Period Investment
Boyd, S., O’Donoghue, B., Mueller, M. T., & Wang, Y. (2013) · Foundations and Trends® in Optimization · doi:10.1561/2400000001 · Cited by 61
Self-citationJoint minimization with alternating Bregman proximity operators
Bauschke, H. H., Combettes, P. L., & Noll, D. (2005) · Cited by 60
Firmly Nonexpansive Mappings and Maximally Monotone Operators: Correspondence and Duality
Bauschke, H. H., Moffat, S. M., & Wang, X. (2011) · Set-Valued and Variational Analysis · doi:10.1007/s11228-011-0187-7 · Cited by 53
Diagonal and Low-Rank Matrix Decompositions, Correlation Matrices, and Ellipsoid Fitting
Saunderson, J., Chandrasekaran, V., Parrilo, P. A., & Willsky, A. S. (2012) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/120872516 · Cited by 48
Message Passing for Dynamic Network Energy ManagementPreprint
Kraning, M., Chu, E., Lavaei, J., & Boyd, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1204.1106 · Cited by 44
Self-citationProximal regularization for online and batch learning
Do, C. B., Le, Q. V., & Foo, C.-S. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553407 · Cited by 44
Reflection-Projection Method for Convex Feasibility Problems with an Obtuse Cone
Bauschke, H. H., & Kruk, S. G. (2004) · Journal of Optimization Theory and Applications · doi:10.1023/b:jota.0000025708.31430.22 · Cited by 43
Multiplier methods for convex programming
Kort, B. W., & Bertsekas, D. P. (1973) · IEEE Conference on Decision and Control including the Symposium on Adaptive Processes (CDC) · doi:10.1109/cdc.1973.269203 · Cited by 40
Polyak, B. T. (1970) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(70)90036-4 · Cited by 39
Fast Projections onto ℓ1,q -Norm Balls for Grouped Feature Selection
Sra, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23808-6_20 · Cited by 38
An Improved Three-Weight Message-Passing AlgorithmPreprint
Derbinsky, N., Bento, J., Elser, V., & Yedidia, J. S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.1961 · Cited by 37
How to project onto an isotone projection cone
Németh, A. B., & Németh, S. Z. (2010) · Linear Algebra and its Applications · doi:10.1016/j.laa.2010.02.008 · Cited by 37
Decomposition methods for large scale LP decoding
Barman, S., Liu, X., Draper, S. C., & Recht, B. (2011) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2011.6120176 · Cited by 35
Penot, J. (1998) · Journal of Approximation Theory · doi:10.1006/jath.1998.3201 · Cited by 34
The higher-order derivatives of spectral functions
Sendov, H. S. (2007) · Linear Algebra and its Applications · doi:10.1016/j.laa.2006.12.013 · Cited by 32
Prox-Regularity of Spectral Functions and Spectral Sets
Daniilidis, A., Lewis, A. S., Malick, J., & Sendov, H. S. (2008) · Journal of convex analysis · doi:10.68381/jca15039 · Cited by 27
Relaxing Tight Frame Condition in Parallel Proximal Methods for Signal Restoration
Pustelnik, N., Pesquet, J.-C., & Chaux, C. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2173684 · Cited by 25
Market Mechanisms and Mathematical Programming
Uzawa, H. (1960) · Econometrica · doi:10.2307/1907569 · Cited by 23
Primal-dual proximal point algorithm for linearly constrained convex programming problems
Ibaraki, S., Fukushima, M., & Ibaraki, T. (1992) · Computational Optimization and Applications · doi:10.1007/bf00253807 · Cited by 18
Nonlinear Proximal Decomposition Method for Convex Programming
Kyono, M., & Fukushima, M. (2000) · Journal of Optimization Theory and Applications · doi:10.1023/a:1004655531273 · Cited by 15
Practical aspects of the Moreau-Yosida regularization I : theoretical properties
Lemaréchal, C., & Sagastizábal, C. A. (1994) · Cited by 15
Stochastic ADMM for Nonsmooth OptimizationPreprint
Ouyang, H. J., He, N., & Gray, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.0632 · Cited by 15
Discussion: Latent variable graphical model selection via convex optimization
Giraud, C., & Tsybakov, A. B. (2012) · The Annals of Statistics · doi:10.1214/12-aos984 · Cited by 15
Forward-backward splitting techniques: theory and applications
Chen, G. H.-G. (1994) · Cited by 14
Risk estimation for matrix recovery with spectral regularizationPreprint
Deledalle, C., Vaiter, S., Gabriel Peyré, A., Fadili, J. M., & Dossal, C. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.1482 · Cited by 11
Locally symmetric submanifolds lift to spectral manifoldsPreprint
Daniilidis, A., Malick, J., & Sendov, H. S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1212.3936 · Cited by 10
Discussion: Latent variable graphical model selection via convex optimization
Candès, E. J., & Soltanolkotabi, M. (2012) · The Annals of Statistics · doi:10.1214/12-aos1001 · Cited by 7
Dual convergence of the proximal point method with Bregman distances for linear programming
Neto, J. X. C., Ferreira, O. P., Iusem, A. N., & Monteiro, R. D. C. (2006) · Optimization methods & software · doi:10.1080/10556780600565083 · Cited by 5
Research works printed but not linked: 37
- Arrow, 1958, Studies in Linear and Nonlinear Programming
- Attouch, 1977, Convergence de fonctions convexes, de sous-différentiels et semi-groupes, C.R Acad. Sei. Paris, 284, 539
- Beck, 2010, Convex Optimization in Signal Processing and Communications, 42
- Bertsekas, 1999, Nonlinear Programming
- Bertsekas, 1989, Parallel and Distributed Computation: Numerical Methods
- Blackford, 1997, ScaLAPACK user’s guide, 10.1137/1.9780898719642 · DOI · Paper page
- Boyd, 2007, Localization and cutting-plane methods
- Candès, 2009, Robust principal component analysis?
- Chandrasekaran, 2009, Allerton 2009, 962
- Daniilidis, 2013, Orthogonal invariance and identifiability
- Demmel, 1993, Parallel numerical linear algebra, 10.1017/S096249290000235X · doi:10.1017/s096249290000235x
- Dempster, 1972, Covariance selection, Biometrics, 28, 157, 10.2307/2528966 · DOI · Paper page
- Duchi, 2008, Efficient projections onto the Q-ball for learning in high dimensions, 272
- Franklin, 1994, Feedback Control of Dynamic Systems, 10.1115/IMECE1994-0541 · doi:10.1115/imece1994-0541
- Gabay, 1983, Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems
- Goldfarb, 2011, Fast first-order methods for composite convex optimization with line search
- Granas, 2003, Fixed Point Theory, 10.1007/978-0-387-21593-8 · doi:10.1007/978-0-387-21593-8
- Lusem, 1999, Augmented Lagrangian methods and proximal point methods for convex optimization, Investigación Operativa, 8, 11
- Jenatton, 2010, Proximal methods for sparse hierarchical dictionary learning
- Kachurovskii, 1960, Monotone operators and convex functionals, Uspekhi Matematicheskikh Nauk, 15, 213
- Kim, 2009, ℓ1 trend filtering, SIAM Review, 51, 339, 10.1137/070690274 · DOI · Paper page
- Lemaire, 1989, The proximal algorithm, 73
- Ma, 2012, Alternating direction methods for latent variable gaussian graphical model selection
- Martinet, 1970, Régularisation d’inéquations variationnelles par approximations successives
- Martinet, 1972, Détermination approchée d’un point fixe d’une application pseudo-contractante, C.R. Acad. Sei. Paris, 274A, 163
- Nedic, 2010, Convex Optimization in Signal Processing and Communications
- Nesterov, 1983, A method of solving a convex programming problem with convergence rate O(1/k2), Soviet Mathematics Doklady, 27, 372
- Quattoni, 2009, An efficient projection for ℓ1,∞ regularization, 857
- Rockafellar, 1970, Convex Analysis, 10.1515/9781400873173 · DOI · Paper page
- Rockafellar, 1998, Variational Analysis, 10.1007/978-3-642-02431-3 · DOI · Paper page
- Tseng, 2008, On accelerated proximal gradient methods for convex-concave optimization
- Vandenberghe, 2010, Optimization methods for large-scale systems
- von Neumann, 1937, Some matrix inequalities and metrization of matrix space, Tomsk University Review, 1, 286
- von Neumann, Functional Operators, Volume 2: The Geometry of Orthogonal Spaces
- Yosida, 1968, Functional Analysis, 10.1007/978-3-662-11791-0 · doi:10.1007/978-3-662-11791-0
- Zarantonello, 1960, Solving functional equations by contractive averaging
- Zarantonello, 1971, Projections on convex sets in Hilbert space and spectral theory. I. Projections on convex sets, 237
Cited by
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3070203 · Cited by 2,878
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks
Wang, S., Teng, Y., & Perdikaris, P. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/20m1318043 · Cited by 2,048
Learning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116
Metadata
- Authors
- Parikh, N., & Boyd, S.
- Affiliations
- Stanford University (US)
- Venue
- Foundations and Trends® in Optimization
- Publisher
- Emerald
- Published
- 2014-01-13
- Volume
- 1
- Issue
- 3
- Pages
- 127-239
- ISSN
- 2167-3888 (ISSN-L)
- 2167-3918
- OpenAlex
- W2913535645
- Cited by
- 4,102
- References
- 182
- Citations by year
- 2026: 204 · 2025: 310 · 2024: 331 · 2023: 321 · 2022: 312 · 2021: 432 · 2020: 443 · 2019: 460 · 2018: 407 · 2017: 315 · 2016: 278 · 2015: 189 · 2014: 85
- FWCI
- 299.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Foundations and Trends® in Optimization (publishedVersion)
- now publishers, Inc. eBooks (publishedVersion)
- CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Optimization and Variational Analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Signal Processing › Sparse and Compressive Sensing Techniques
- Keywords
- proximal operator, proximal algorithms, convex optimization, distributed optimization, nonsmooth optimization, large-scale optimization, high-dimensional data, projection onto convex sets, iterative optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.