Randomized smoothing for (parallel) stochastic optimization
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
By combining randomized smoothing techniques with accelerated gradient methods, we obtain convergence rates for stochastic optimization procedures, both in expectation and with high probability, that have optimal dependence on the variance of the gradient estimates. To the best of our knowledge, these are the first variance-based rates for non-smooth optimization. A combination of our techniques with recent work on decentralized optimization yields order-optimal parallel stochastic optimization algorithms. We give applications of our results to several statistical machine learning problems, providing experimental results (in the full version of the paper) demonstrating the effectiveness of our algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Huber, P. J. (1981) · Wiley Series in Probability and Statistics · doi:10.1002/0471725250 · Cited by 7,066
Simulation and the Monte Carlo Method
Rubinstein, R. Y., & Kroese, D. P. (2016) · Wiley Series in Probability and Statistics · doi:10.1002/9781118631980 · Cited by 3,356 · cited 1×
“, [2, 10, 15, 11, 22]).”
An optimal method for stochastic composite optimization
Lan, G. (2012) · Mathematical Programming · doi:10.1007/s10107-010-0434-y · Cited by 427 · cited 4×
“When the objective function f is smooth, meaning that it has Lipschitz continuous gradient, recent work by Juditsky et al. [12] and Lan [13] have shown that if the variance of a stochastic gradient estimate is at most σ2 then the resulting stochastic optimization procedure has convergence rate O(σ/ √ T ).”
MethodInfluential
Cited by
Predicting cancer outcomes from histology and genomics using convolutional networks
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717139115 · Cited by 1,084
MOLI: multi-omics late integration with deep neural networks for drug response prediction
Sharifi-Noghabi, H., Zolotareva, O., Collins, C. C., & Ester, M. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz318 · Cited by 421
Huang, Y.-F., Gulko, B., & Siepel, A. (2017) · Nature Genetics · doi:10.1038/ng.3810 · Cited by 344
A unified framework for stochastic optimization
Powell, W. B. (2019) · European Journal of Operational Research · doi:10.1016/j.ejor.2018.07.014 · Cited by 314
Domingos, E., Ojeme, B., & Daramola, O. (2021) · Computation · doi:10.3390/computation9030034 · Cited by 77
“This allowed better segmentation of the data because the neurons were much more trained [47].”
MethodA murine aging cell atlas reveals cell identity and tissue-specific trajectories of aging
(2019) · doi:10.1101/657726 · Cited by 12
Deep Genomic Signature for early metastasis prediction in prostate cancer
Sharifi-Noghabi, H., Liu, Y., Erho, N., Shrestha, R., Alshalalfa, M., Davicioni, E., Collins, C. C., & Ester, M. (2018) · doi:10.1101/276055 · Cited by 11
Metadata
- Authors
- Duchi, J. C., Bartlett, P. L., & Wainwright, M. J.
- Venue
- 2012 IEEE 51st IEEE Conference on Decision and Control (CDC)
- Publisher
- IEEE
- Published
- 2012-12
- Pages
- 5442-5444
- OpenAlex
- W2002517365
- Cited by
- 96
- References
- 5
- FWCI
- 0.98
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.