The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Hybrid Deterministic-Stochastic Methods for Data Fitting
Open accessAbstract
Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum; these methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate-of-convergence analysis shows that by controlling the sample size in an incremental-gradient algorithm, it is possible to maintain the steady convergence rates of full-gradient methods. We detail a practical quasi-Newton implementation based on this approach. Numerical experiments illustrate its potential benefits.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 1×
“…a kk i and associated parameters x kk (cid:48) , and defines the probability of a configuration as The normalizing constant Z i is again set so that the distribution sums to one, and it can be computed using a variant on the forward-backward algorithm used in hidden Markov models [30, § III].”
MethodRobust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
Statistical Analysis of Non-Lattice Data
Besag, J. (1975) · The Statistician · doi:10.2307/2987782 · Cited by 1,559
Mirror descent and nonlinear projected subgradient methods for convex optimization
Beck, A., & Teboulle, M. (2003) · Operations Research Letters · doi:10.1016/s0167-6377(02)00231-6 · Cited by 1,060
Concentration Inequalities and Model Selection
(2007) · Lecture Notes in Mathematics · doi:10.1007/978-3-540-48503-2 · Cited by 744
Error bounds and convergence analysis of feasible descent methods: a general approach
Luo, Z.-Q., & Tseng, P. (1993) · Annals of Operations Research · doi:10.1007/bf02096261 · Cited by 420
Quasi-Random Sequences and Their Discrepancies
Morokoff, W. J., & Caflisch, R. E. (1994) · SIAM Journal on Scientific Computing · doi:10.1137/0915077 · Cited by 377
A New Class of Incremental Gradient Methods for Least Squares Problems
Bertsekas, D. P. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623495287022 · Cited by 312
A Convergent Incremental Gradient Method with a Constant Step Size
Blatt, D., Hero, A. O., & Gauchman, H. (2007) · SIAM Journal on Optimization · doi:10.1137/040615961 · Cited by 306
On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning
Byrd, R. H., Chin, G. M., Neveitt, W., & Nocedal, J. (2011) · SIAM Journal on Optimization · doi:10.1137/10079923x · Cited by 242
Poyiadjis, G., Doucet, A., & Singh, S. S. (2011) · Biometrika · doi:10.1093/biomet/asq062 · Cited by 235
Smooth Optimization with Approximate Gradient
d'Aspremont, A. (2008) · SIAM Journal on Optimization · doi:10.1137/060676386 · Cited by 168
Cited by
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547
A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547
Minimizing finite sums with the stochastic average gradient
Schmidt, M., Le Roux, N., & Bach, F. (2017) · Mathematical Programming · doi:10.1007/s10107-016-1030-6 · Cited by 922
Self-citationAdvances in Variational Inference
Zhang, C., Butepage, J., Kjellstrom, H., & Mandt, S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889774 · Cited by 671
Metadata
- Authors
- Friedlander, M. P., & Schmidt, M.
- Venue
- SIAM Journal on Scientific Computing
- Published
- 2012-01
- Volume
- 34
- Issue
- 3
- Pages
- A1380-A1405
- ISSN
- 1064-8275 (ISSN-L)
- 1095-7197
- OpenAlex
- W1751687266
- Cited by
- 254
- References
- 12
- FWCI
- 6.65
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.