Proximal Algorithms
The DOI matches its Crossref record; 180 of the 182 references it cites were found in a registry or an index.
Abstract
This monograph is about a class of optimization algorithms called proximal algorithms. Much like Newton’s method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, large-scale, or distributed versions of these problems. They are very generally applicable, but are especially well-suited to problems of substantial recent interest involving large or high-dimensional datasets. Proximal methods sit at a higher level of abstraction than classical algorithms like Newton’s method: the base operation is evaluating the proximal operator of a function, which itself involves solving a small convex optimization problem. These subproblems, which generalize the problem of projecting a point onto a convex set, often admit closed-form solutions or can be solved very quickly with standard or simple specialized methods. Here, we discuss the many different interpretations of proximal operators and algorithms, describe their connections to many other topics in optimization and applied mathematics, survey some popular algorithms, and provide a large number of examples of proximal operators that commonly arise in practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Self-citationRegression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Self-citation(2021) · A First Course in Statistical Programming with R · doi:10.1017/9781108993456.011 · Cited by 13,724
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Self-citationA method for the solution of certain non-linear problems in least squares
Levenberg, K. (1944) · Quarterly of Applied Mathematics · doi:10.1090/qam/10666 · Cited by 12,359
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
De-noising by soft-thresholding
Donoho, D. L. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382009 · Cited by 9,619
Research works printed but not linked: 37
- Arrow, 1958, Studies in Linear and Nonlinear Programming
- Attouch, 1977, Convergence de fonctions convexes, de sous-différentiels et semi-groupes, C.R Acad. Sei. Paris, 284, 539
- Beck, 2010, Convex Optimization in Signal Processing and Communications, 42
- Bertsekas, 1999, Nonlinear Programming
- Bertsekas, 1989, Parallel and Distributed Computation: Numerical Methods
- Blackford, 1997, ScaLAPACK user’s guide, 10.1137/1.9780898719642 · DOI · Paper page
- Boyd, 2007, Localization and cutting-plane methods
- Candès, 2009, Robust principal component analysis?
- Chandrasekaran, 2009, Allerton 2009, 962
- Daniilidis, 2013, Orthogonal invariance and identifiability
- Demmel, 1993, Parallel numerical linear algebra, 10.1017/S096249290000235X · doi:10.1017/s096249290000235x
- Dempster, 1972, Covariance selection, Biometrics, 28, 157, 10.2307/2528966 · DOI · Paper page
- Duchi, 2008, Efficient projections onto the Q-ball for learning in high dimensions, 272
- Franklin, 1994, Feedback Control of Dynamic Systems, 10.1115/IMECE1994-0541 · doi:10.1115/imece1994-0541
- Gabay, 1983, Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems
- Goldfarb, 2011, Fast first-order methods for composite convex optimization with line search
- Granas, 2003, Fixed Point Theory, 10.1007/978-0-387-21593-8 · doi:10.1007/978-0-387-21593-8
- Lusem, 1999, Augmented Lagrangian methods and proximal point methods for convex optimization, Investigación Operativa, 8, 11
- Jenatton, 2010, Proximal methods for sparse hierarchical dictionary learning
- Kachurovskii, 1960, Monotone operators and convex functionals, Uspekhi Matematicheskikh Nauk, 15, 213
- Kim, 2009, ℓ1 trend filtering, SIAM Review, 51, 339, 10.1137/070690274 · DOI · Paper page
- Lemaire, 1989, The proximal algorithm, 73
- Ma, 2012, Alternating direction methods for latent variable gaussian graphical model selection
- Martinet, 1970, Régularisation d’inéquations variationnelles par approximations successives
- Martinet, 1972, Détermination approchée d’un point fixe d’une application pseudo-contractante, C.R. Acad. Sei. Paris, 274A, 163
- Nedic, 2010, Convex Optimization in Signal Processing and Communications
- Nesterov, 1983, A method of solving a convex programming problem with convergence rate O(1/k2), Soviet Mathematics Doklady, 27, 372
- Quattoni, 2009, An efficient projection for ℓ1,∞ regularization, 857
- Rockafellar, 1970, Convex Analysis, 10.1515/9781400873173 · DOI · Paper page
- Rockafellar, 1998, Variational Analysis, 10.1007/978-3-642-02431-3 · DOI · Paper page
- Tseng, 2008, On accelerated proximal gradient methods for convex-concave optimization
- Vandenberghe, 2010, Optimization methods for large-scale systems
- von Neumann, 1937, Some matrix inequalities and metrization of matrix space, Tomsk University Review, 1, 286
- von Neumann, Functional Operators, Volume 2: The Geometry of Orthogonal Spaces
- Yosida, 1968, Functional Analysis, 10.1007/978-3-662-11791-0 · doi:10.1007/978-3-662-11791-0
- Zarantonello, 1960, Solving functional equations by contractive averaging
- Zarantonello, 1971, Projections on convex sets in Hilbert space and spectral theory. I. Projections on convex sets, 237
Cited by
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3070203 · Cited by 2,878
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks
Wang, S., Teng, Y., & Perdikaris, P. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/20m1318043 · Cited by 2,048
Learning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning
Sun, Y., Babu, P., & Palomar, D. P. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2601299 · Cited by 1,820
Tensor Decomposition for Signal Processing and Machine Learning
Sidiropoulos, N. D., De Lathauwer, L., Fu, X., Huang, K., Papalexakis, E. E., & Faloutsos, C. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2690524 · Cited by 1,617
Plug-and-Play Image Restoration With Deep Denoiser Prior
Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3088914 · Cited by 1,113
A Survey of Sparse Representation: Algorithms and Applications
Zhang, Z., Xu, Y., Yang, J., Li, X., & Zhang, D. (2015) · IEEE Access · doi:10.1109/access.2015.2430359 · Cited by 1,092
The Little Engine That Could: Regularization by Denoising (RED)
Romano, Y., Elad, M., & Milanfar, P. (2017) · SIAM Journal on Imaging Sciences · doi:10.1137/16m1102884 · Cited by 934
“(1)In the context of optimization, a smaller class of the general denoising algorithms we define are characterized as “proximal operators” [61].”
BackgroundA survey of distributed optimization
Yang, T., Yi, X., Wu, J., Yuan, Y., Wu, D., Meng, Z., Hong, Y., Wang, H., Lin, Z., & Johansson, K. H. (2019) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2019.05.006 · Cited by 913
Smart “Predict, then Optimize”
Elmachtoub, A. N., & Grigas, P. (2022) · Management Science · doi:10.1287/mnsc.2020.3922 · Cited by 773
Distributionally Robust Stochastic Optimization with Wasserstein Distance
Gao, R., & Kleywegt, A. (2023) · Mathematics of Operations Research · doi:10.1287/moor.2022.1275 · Cited by 630
Problem formulations and solvers in linear SVM: a review
Chauhan, V. K., Dahiya, K., & Sharma, A. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-018-9614-6 · Cited by 660
Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding
O’Donoghue, B., Chu, E., Parikh, N., & Boyd, S. (2016) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-016-0892-3 · Cited by 664
“See Esser et al. [63] for a detailed discussion of various first-order methods and the relationships between them, and Parikh and Boyd [64] for a survey of proximal algorithms in particular.”
MethodSelf-citationEAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples
Chen, P.-Y., Sharma, Y., Zhang, H., Yi, J., & Hsieh, C.-J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11302 · Cited by 569
Deep Learning Techniques for Inverse Problems in Imaging
Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R. G., Dimakis, A. G., & Willett, R. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991563 · Cited by 577
Data-Driven Sparse Structure Selection for Deep Neural Networks
Huang, Z., & Wang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_19 · Cited by 549
DeconvolutionLab2: An open-source software for deconvolution microscopy
Sage, D., Donati, L., Soulez, F., Fortun, D., Schmit, G., Seitz, A., Guiet, R., Vonesch, C., & Unser, M. (2017) · Methods · doi:10.1016/j.ymeth.2016.12.015 · Cited by 600
Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks
Zhao, H., Yao, Q., Li, J., Song, Y., & Lee, D. L. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098063 · Cited by 565
An introduction to continuous optimization for imaging
Chambolle, A., & Pock, T. (2016) · Acta Numerica · doi:10.1017/s096249291600009x · Cited by 545
Certifying Some Distributional Robustness with Principled Adversarial Training
Sinha, A., Namkoong, H., Volpi, R., & Duchi, J. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10571 · Cited by 307
Low rank subspace clustering (LRSC)
Vidal, R., & Favaro, P. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2013.08.006 · Cited by 482
Communication Efficient Distributed Machine Learning with the Parameter Server
Li, M., Andersen, D. G., Smola, A. J., & Yu, K. (2014) · neural information processing systems · Cited by 715
Further improvements to the ptychographical iterative engine
Maiden, A., Johnson, D., & Li, P. (2017) · Optica · doi:10.1364/optica.4.000736 · Cited by 452
Multi-Agent Distributed Optimization via Inexact Consensus ADMM
Chang, T.-H., Hong, M., & Wang, X. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2367458 · Cited by 444
An Overview of Low-Rank Matrix Recovery From Incomplete Observations
Davenport, M. A., & Romberg, J. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2539100 · Cited by 439
Sparse Regularization via Convex Analysis
Selesnick, I. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2711501 · Cited by 429
Komodakis, N., & Pesquet, J.-C. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2377273 · Cited by 419
Stochastic Variance Reduction for Nonconvex Optimization
Reddi, S. J., Hefny, A., Sra, S., Póczos, B., & Smola, A. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06160 · Cited by 215
Modern regularization methods for inverse problems
Benning, M., & Burger, M. (2018) · Acta Numerica · doi:10.1017/s0962492918000016 · Cited by 412
Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution
Shen, G., Jia, J., Nie, L., Feng, F., Zhang, C., Hu, T., Chua, T.-S., & Zhu, W. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/536 · Cited by 383
Learning Collaborative Sparse Representation for Grayscale-Thermal Tracking
Li, C., Cheng, H., Hu, S., Liu, X., Tang, J., & Lin, L. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2614135 · Cited by 391
Structured Pruning for Deep Convolutional Neural Networks: A Survey
He, Y., & Xiao, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3334614 · Cited by 363
Jiang, H., He, P., Chen, W., Liu, X., Gao, J., & Zhao, T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.197 · Cited by 162
Non-local Color Image Denoising with Convolutional Neural Networks
Lefkimmiatis, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.623 · Cited by 363
Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning
Kuhn, D., Esfahani, P. M., Nguyen, V. A., & Shafieezadeh-Abadeh, S. (2019) · Operations Research & Management Science in the Age of Analytics · doi:10.1287/educ.2019.0198 · Cited by 342
Tank, A., Covert, I., Foti, N., Shojaie, A., & Fox, E. B. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3065601 · Cited by 282
Train faster, generalize better: Stability of stochastic gradient descent
Hardt, M., Recht, B., & Singer, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01240 · Cited by 323
Quantized Precoding for Massive MU-MIMO
Jacobsson, S., Durisi, G., Coldrey, M., Goldstein, T., & Studer, C. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2723000 · Cited by 367
Nonlinear Chirp Mode Decomposition: A Variational Method
Chen, S., Dong, X., Peng, Z., Zhang, W., & Meng, G. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2731300 · Cited by 349
Zero-Shot Learning via Joint Latent Similarity Embedding
Zhang, Z., & Saligrama, V. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.649 · Cited by 336
Behavioral systems theory in data-driven analysis, signal processing, and control
Markovsky, I., & Dörfler, F. (2021) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2021.09.005 · Cited by 310
A comprehensive survey on regularization strategies in machine learning
Tian, Y., & Zhang, Y. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.005 · Cited by 298
Universal Denoising Networks : A Novel CNN Architecture for Image Denoising
Lefkimmiatis, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00338 · Cited by 300
Wang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07576 · Cited by 146
An electroencephalographic signature predicts antidepressant response in major depression
Wu, W., Zhang, Y., Jiang, J., Lucas, M. V., Fonzo, G. A., Rolle, C. E., Cooper, C., Chin-Fatt, C., Krepel, N., Cornelssen, C. A., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-019-0397-3 · Cited by 302
FlexISP: a flexible camera image processing framework
Heide, F., Steinberger, M., Tsai, Y.-T., Rouf, M., Pająk, D., Reddy, D., Gallo, O., Liu, J., Heidrich, W., Egiazarian, K., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661260 · Cited by 298
Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data
Hallac, D., Vare, S., Boyd, S., & Leskovec, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098060 · Cited by 252
Self-citationFast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
Attouch, H., Chbani, Z., Peypouquet, J., & Redont, P. (2018) · Mathematical Programming · doi:10.1007/s10107-016-0992-8 · Cited by 277
Ebner, M., Wang, G., Li, W., Aertsen, M., Patel, P. A., Aughwane, R., Melbourne, A., Doel, T., Dymarkowski, S., De Coppi, P., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116324 · Cited by 288
Data-Driven Aerospace Engineering: Reframing the Industry with Machine Learning
Brunton, S. L., Nathan Kutz, J., Manohar, K., Aravkin, A. Y., Morgansen, K., Klemisch, J., Goebel, N., Buttrick, J., Poskin, J., Blom-Schieber, A. W., et al. (2021) · AIAA Journal · doi:10.2514/1.j060131 · Cited by 270
Udell, M., Horn, C., Zadeh, R., & Boyd, S. (2016) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000055 · Cited by 245
Self-citationReal-Time Power System State Estimation and Forecasting via Deep Unrolled Neural Networks
Zhang, L., Wang, G., & Giannakis, G. B. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2926023 · Cited by 257
Deep Unfolding for Communications Systems: A Survey and Some New Directions
Balatsoukas-Stimming, A., & Studer, C. (2019) · 2019 IEEE International Workshop on Signal Processing Systems (SiPS) · doi:10.1109/sips47522.2019.9020494 · Cited by 266
SLOPE—Adaptive variable selection via convex optimization
Bogdan, M., van den Berg, E., Sabatti, C., Su, W., & Candès, E. J. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas842 · Cited by 277
Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
Konecny, J., Liu, J., Richtarik, P., & Takac, M. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2015.2505682 · Cited by 254
Non-local Meets Global: An Integrated Paradigm for Hyperspectral Image Restoration
He, W., Yao, Q., Li, C., Yokoya, N., Zhao, Q., Zhang, H., & Zhang, L. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3027563 · Cited by 199
Model-Guided Deep Hyperspectral Image Super-Resolution
Dong, W., Zhou, C., Wu, F., Wu, J., Shi, G., & Li, X. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3078058 · Cited by 254
Plug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery
Ahmad, R., Bouman, C. A., Buzzard, G. T., Chan, S., Liu, S., Reehorst, E. T., & Schniter, P. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2949470 · Cited by 261
Fast and flexible convolutional sparse coding
Heide, F., Heidrich, W., & Wetzstein, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299149 · Cited by 245
Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers
Tachella, J., Altmann, Y., Mellado, N., McCarthy, A., Tobin, R., Buller, G. S., Tourneret, J.-Y., & McLaughlin, S. (2019) · Nature Communications · doi:10.1038/s41467-019-12943-7 · Cited by 245
Accelerated Methods for NonConvex Optimization
Carmon, Y., Duchi, J. C., Hinder, O., & Sidford, A. (2018) · SIAM Journal on Optimization · doi:10.1137/17m1114296 · Cited by 190
Kamilov, U. S., Bouman, C. A., Buzzard, G. T., & Wohlberg, B. (2023) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3199595 · Cited by 244
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
Xie, X., Zhou, P., Li, H., Lin, Z., & Yan, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3423382 · Cited by 220
Regularization by Denoising: Clarifications and New Interpretations
Reehorst, E. T., & Schniter, P. (2019) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2018.2880326 · Cited by 239
Denoising Diffusion Models for Plug-and-Play Image Restoration
Zhu, Y., Zhang, K., Liang, J., Cao, J., Wen, B., Timofte, R., & Gool, L. V. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00129 · Cited by 228
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery
Vowels, M. J., Camgoz, N. C., & Bowden, R. (2023) · ACM Computing Surveys · doi:10.1145/3527154 · Cited by 237
“Firstly, Varando (2020) [251] proposes a proximal gradient [180] optimization objective that yields a linear SEM and corresponding DAG.”
BackgroundA review of physics-based machine learning in civil engineering
Vadyala, S. R., Betgeri, S. N., Matthews, J. C., & Matthews, E. (2022) · Results in Engineering · doi:10.1016/j.rineng.2021.100316 · Cited by 235
Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMM
Giselsson, P., & Boyd, S. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2016.2564160 · Cited by 225
Self-citationNetwork Lasso: Clustering and Optimization in Large Graphs
Hallac, D., Leskovec, J., & Boyd, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783313 · Cited by 235
Self-citationTrainable ISTA for Sparse Signal Recovery
Ito, D., Takabe, S., & Wadayama, T. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2912879 · Cited by 223
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
Li, Y., Gu, S., Mayer, C., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00804 · Cited by 208
Movement pruning: Adaptive sparsity by fine-tuning
Sanh, V., Wolf, T., & Rush, A. M. (2020) · arXiv · doi:10.48550/arxiv.2005.07683 · Cited by 218
“We compare our results against several state-of-the-art pruning baselines: Reweighted Proximal Pruning (RPP) [Guo et al., 2019] combines re-weighted L 1 minimization and Proximal Projection [Parikh and Boyd, 2014] to perform unstructured pruning.”
ResultHyperparameter optimization with approximate gradient
Pedregosa, F. (2016) · arXiv · doi:10.48550/arxiv.1602.02355 · Cited by 217
A Machine Learning Approach for Air Quality Prediction: Model Regularization and Optimization
Zhu, D., Cai, C., Yang, T., & Zhou, X. (2018) · Big Data and Cognitive Computing · doi:10.3390/bdcc2010005 · Cited by 213
Convolutional Dictionary Learning: A Comparative Review and New Algorithms
Garcia-Cardona, C., & Wohlberg, B. (2018) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2018.2840334 · Cited by 216
An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
Sun, Y., Wohlberg, B., & Kamilov, U. S. (2019) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2019.2893568 · Cited by 207
A Survey on Canonical Correlation Analysis
Yang, X., Liu, W., Liu, W., & Tao, D. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2958342 · Cited by 202
An Augmented Lagrangian Based Algorithm for Distributed NonConvex Optimization
Houska, B., Frasch, J., & Diehl, M. (2016) · SIAM Journal on Optimization · doi:10.1137/140975991 · Cited by 197
Channel Charting: Locating Users Within the Radio Environment Using Channel State Information
Studer, C., Medjkouh, S., Gonultas, E., Goldstein, T., & Tirkkonen, O. (2018) · IEEE Access · doi:10.1109/access.2018.2866979 · Cited by 203
Neumann Networks for Linear Inverse Problems in Imaging
Gilton, D., Ongie, G., & Willett, R. (2020) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2019.2948732 · Cited by 178
A Universal Catalyst for First-Order Optimization
Lin, H., Mairal, J., & Harchaoui, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02186 · Cited by 165
A Review on Deep Learning in Medical Image Reconstruction
Zhang, H.-M., & Dong, B. (2020) · Journal of the Operations Research Society of China · doi:10.1007/s40305-019-00287-4 · Cited by 191
A Unified Framework for Sparse Relaxed Regularized Regression: SR3
Zheng, P., Askham, T., Brunton, S. L., Kutz, J. N., & Aravkin, A. Y. (2019) · IEEE Access · doi:10.1109/access.2018.2886528 · Cited by 193
Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization
Shlezinger, N., Eldar, Y. C., & Boyd, S. P. (2022) · IEEE Access · doi:10.1109/access.2022.3218802 · Cited by 185
Self-citationUnsupervised Feature Learning via Non-Parametric Instance-level Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01978 · Cited by 163
Graph-Based Blind Image Deblurring From a Single Photograph
Bai, Y., Cheung, G., Liu, X., & Gao, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2874290 · Cited by 165
Network Inference via the Time-Varying Graphical Lasso
Hallac, D., Park, Y., Boyd, S., & Leskovec, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098037 · Cited by 174
Self-citationSparseDGCNN: Recognizing Emotion From Multichannel EEG Signals
Zhang, G., Yu, M., Liu, Y.-J., Zhao, G., Zhang, D., & Zheng, W. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3051332 · Cited by 185
Bayesian computation: a summary of the current state, and samples backwards and forwards
Green, P. J., Łatuszyński, K., Pereyra, M., & Robert, C. P. (2015) · Statistics and Computing · doi:10.1007/s11222-015-9574-5 · Cited by 199
Bolt-on Differential Privacy for Scalable Stochastic Gradient Descent-based Analytics
Wu, X., Li, F., Kumar, A., Chaudhuri, K., Jha, S., & Naughton, J. (2017) · Proceedings of the 2017 ACM International Conference on Management of Data · doi:10.1145/3035918.3064047 · Cited by 175
Shah, S. A., & Koltun, V. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1700770114 · Cited by 181
Low rank matrix recovery from rank one measurements
Kueng, R., Rauhut, H., & Terstiege, U. (2017) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2015.07.007 · Cited by 173
Liu, Y., Zhan, Z., Cai, J.-F., Guo, D., Chen, Z., & Qu, X. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2550080 · Cited by 177
Convolutional Sparse and Low-Rank Coding-Based Rain Streak Removal
Zhang, H., & Patel, V. M. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.145 · Cited by 165
Anderson, J., Doyle, J. C., Low, S. H., & Matni, N. (2019) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2019.03.006 · Cited by 166
Heide, F., Xiao, L., Heidrich, W., & Hullin, M. B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.418 · Cited by 166
A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization
Huang, K., Sidiropoulos, N. D., & Liavas, A. P. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2576427 · Cited by 172
Udell, M., Horn, C., Zadeh, R., & Boyd, S. (2016) · now publishers, Inc. eBooks · doi:10.1561/9781680831412 · Cited by 161
Self-citation
Metadata
- Authors
- Parikh, N., & Boyd, S.
- Affiliations
- Stanford University (US)
- Venue
- Foundations and Trends® in Optimization
- Publisher
- Emerald
- Published
- 2014-01-13
- Volume
- 1
- Issue
- 3
- Pages
- 127-239
- ISSN
- 2167-3888 (ISSN-L)
- 2167-3918
- OpenAlex
- W2913535645
- Cited by
- 4,102
- References
- 182
- Citations by year
- 2026: 204 · 2025: 310 · 2024: 331 · 2023: 321 · 2022: 312 · 2021: 432 · 2020: 443 · 2019: 460 · 2018: 407 · 2017: 315 · 2016: 278 · 2015: 189 · 2014: 85
- FWCI
- 299.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Foundations and Trends® in Optimization (publishedVersion)
- now publishers, Inc. eBooks (publishedVersion)
- CERN Document Server (European Organization for Nuclear Research) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Optimization and Variational Analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Signal Processing › Sparse and Compressive Sensing Techniques
- Keywords
- proximal operator, proximal algorithms, convex optimization, distributed optimization, nonsmooth optimization, large-scale optimization, high-dimensional data, projection onto convex sets, iterative optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.