A Universal Catalyst for First-Order Optimization
Open accessThe DOI matches its OpenAlex record; 25 of the 25 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
Parikh, N., & Boyd, S. (2014) · Foundations and Trends® in Optimization · doi:10.1561/2400000003 · Cited by 4,102 · cited 1×
“The catalyst acceleration can be interpreted as a variant of the proximal point algorithm [3, 9], which is a central concept in convex optimization, underlying augmented Lagrangian approaches, and composite minimization schemes [5, 20].”
BackgroundIntroductory Lectures on Convex Optimization: A Basic Course
Nesterov, Y. E. (2014) · Medical Entomology and Zoology · Cited by 3,557
Convex Analysis and Monotone Operator Theory in Hilbert Spaces
Bauschke, H. H., & Combettes, P. L. (2017) · CMS Books in Mathematics · doi:10.1007/978-3-319-48311-5 · Cited by 3,307 · cited 1×
“The catalyst acceleration can be interpreted as a variant of the proximal point algorithm [3, 9], which is a central concept in convex optimizati on, underlying augmented Lagrangian approaches, and composite minimization schemes [5, 20].”
BackgroundConvex Analysis and Minimization Algorithms I
Hiriart-Urruty, J.-B., & Lemaréchal, C. (1993) · Grundlehren der mathematischen Wissenschaften · doi:10.1007/978-3-662-02796-7 · Cited by 2,378 · cited 2×
“[11] J.”
BackgroundRobust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
Gradient methods for minimizing composite functions
Nesterov, Y. (2013) · Mathematical Programming · doi:10.1007/s10107-012-0629-5 · Cited by 1,298 · cited 3×
“Most of the algorithms we consider here, namely the proximal gr dient method [4, 19], SAGA [6], (Prox)-SVRG [27], can handle compo site objectives with a regularization penaltyψ that admits a proximal operator p oxψ, defined for anyz as proxψ(z) , argmin y∈Rp { ψ(y) + 1 2 ‖y − z‖(2) } .”
BackgroundMethodEfficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems
Nesterov, Y. (2012) · SIAM Journal on Optimization · doi:10.1137/100802001 · Cited by 1,187 · cited 1×
“This is the case for full gradient [4, 19] and block coordinate descent methods [18, 21], which already have well-know n accelerated variants.”
BackgroundMinimizing finite sums with the stochastic average gradient
Schmidt, M., Le Roux, N., & Bach, F. (2017) · Mathematical Programming · doi:10.1007/s10107-016-1030-6 · Cited by 922 · cited 6×
“More importantly, it also applies to incremental algorithms such as SA G [24], SAGA [6], Finito/MISO [7, 14], SDCA [25], and SVRG [27].”
BackgroundMethodInfluentialSAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
Defazio, A., Bach, F., & Lacoste-Julien, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.0202 · Cited by 834 · cited 6×
“1 ε ) ) SAGA [6] Finito/MISO-Prox not avail.”
BackgroundMethodInfluentialOptimization with Sparsity-Inducing Penalties
Bach, F., Jenatton, R., Mairal, J., & Obozinski, G. (2025) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000015 · Cited by 730 · cited 1×
“The functionψ in (1) acts as a regularizer; it is typically chosen to be the square d l2-norm, which is smooth, or to be a non-differentiable penalty such as the l1-norm or another sparsity-inducing norm [2].”
BackgroundSelf-citationConvex Optimization Algorithms
Bertsekas, D. P. (2015) · Cited by 689
Richtárik, P., & Takáč, M. (2014) · Mathematical Programming · doi:10.1007/s10107-012-0614-z · Cited by 654 · cited 3×
“We first show how to accelerate full gradient and randomized c oordinate descent algorithms [21].”
BackgroundMethodResultOptimization with Sparsity-Inducing Penalties
Bach, F. (2011) · now publishers, Inc. eBooks · doi:10.1561/9781601985118 · Cited by 608
A Proximal Stochastic Gradient Method with Progressive Variance Reduction
Xiao, L., & Zhang, T. (2014) · SIAM Journal on Optimization · doi:10.1137/140961791 · Cited by 592 · cited 13×
“More importantly, it also applies to incremental algorithms such as SA G [24], SAGA [6], Finito/MISO [7, 14], SDCA [25], and SVRG [27].”
BackgroundMethodInfluentialAccelerated proximal stochastic dual coordinate ascent for regularized loss minimization
Shalev-Shwartz, S., & Zhang, T. (2016) · Mathematical Programming · doi:10.1007/s10107-014-0839-0 · Cited by 360 · cited 4×
“Specifically, we use the concept of estimate sequence from Nesterov [17], whereas the direct proof of [26], in the context of SDCA, does not extend to non-strongly convex objectives.”
BackgroundMethodInfluentialConvergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization
Schmidt, M., Roux, N. L., & Bach, F. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1109.2415 · Cited by 237 · cited 2×
“Another useful methodological contribution was the convergence analysis of inexact proximal gradient methods of [23].”
MethodIncremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning
Mairal, J. (2015) · SIAM Journal on Optimization · doi:10.1137/140957639 · Cited by 232 · cited 18×
“More importantly, it also applies to incremental algorithms such as SA G [24], SAGA [6], Finito/MISO [7, 14], SDCA [25], and SVRG [27].”
BackgroundMethodInfluentialSelf-citationNew Proximal Point Algorithms for Convex Minimization
Güler, O. (1992) · SIAM Journal on Optimization · doi:10.1137/0802032 · Cited by 231 · cited 9×
“This is where our approach differs from the classical proximal po int algorithm of [9].”
BackgroundMethodInfluentialAn optimal randomized incremental gradient method
Lan, G., & Zhou, Y. (2017) · Mathematical Programming · doi:10.1007/s10107-017-1173-0 · Cited by 214 · cited 1×
“Another independent work with related results is [13], which introduce an accelerated method for the minimization of finite sums, which is not based on the proximal point algorithm.”
MethodInexact and Accelerated Proximal Point Algorithms
Salzo, S., & Villa, S. (2012) · Journal of convex analysis · doi:10.68381/jca19062 · Cited by 150 · cited 3×
“(1)Note that our inexact criterion was also studied, among othe rs, in [22], but the analysis of [22] led to the conjecture that this criterion was too weak to warrant accel eration.”
MethodResultFinito: A Faster, Permutable Incremental Gradient Method for Big Data ProblemsPreprint
Defazio, A., Caetano, T. S., & Domke, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.2710 · Cited by 126 · cited 18×
“More importantly, it also applies to incremental algorithms such as SA G [24], SAGA [6], Finito/MISO [7, 14], SDCA [25], and SVRG [27].”
BackgroundMethodInfluentialProximal Stochastic Dual Coordinate AscentPreprint
Shalev‐Shwartz, S., & Zhang, T. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.2717 · Cited by 99 · cited 5×
“The resulting algorithm can be interpreted as a variant of proximal SDCA [25] with a different step size and a more practical optimality certificate—that is, checking the optimality condition does not require evaluating a dual objective.”
BackgroundMethodInfluentialA Lower Bound for the Optimization of Finite SumsPreprint
Agarwal, A., & Bottou, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0723 · Cited by 72 · cited 1×
“Otherwise, their complexity becomes O(n log(1/ε)), which is independent of the condition number and seems theoretically optimal [1].”
BackgroundFrostig, R., Ge, R., Kakade, S. M., & Sidford, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.07512 · Cited by 70
Cited by
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
Metadata
- Authors
- Lin, H., Mairal, J., & Harchaoui, Z.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2015-06-06
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1506.02186
- OpenAlex
- W584312462
- Cited by
- 165
- References
- 25
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.