Variance-Reduced Stochastic Learning by Networked Agents Under Random Reshuffling
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
This paper develops a distributed variance-reduced strategy for a collection of interacting agents that are connected by a graph topology. The resulting diffusion-AVRG (where AVRG stands for “amortized variance-reduced gradient”) algorithm is shown to have linear convergence to the exact solution, and is more memory efficient than other alternative algorithms. When a batch implementation is employed, it is observed in simulations that diffusion-AVRG is more computationally efficient than exact diffusion or EXTRA, while maintaining almost the same communication efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · cited 1×
“For most traditional machine learning tasks, the training data are usually stored at a single computing unit [2]–[5].”
BackgroundDistributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719 · cited 1×
“One can resort to replacing the true gradient by a stochastic gradient approximation, as is commonplace in traditional diffusion or consensus algorithms [12], [13], [25]–[30].”
BackgroundEXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
Shi, W., Ling, Q., Wu, G., & Yin, W. (2015) · SIAM Journal on Optimization · doi:10.1137/14096668x · Cited by 1,261
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs
Nedić, A., Olshevsky, A., & Shi, W. (2017) · SIAM Journal on Optimization · doi:10.1137/16m1084316 · Cited by 1,069 · cited 8×
“The exact diffusion method, in particular, has been shown to have a wider stability range than EXTRA implementations (i.e., it is stable for a wider range of step-sizes, μ), and is also more efficient in terms of communications than DIGing.”
MethodInfluentialOn the Linear Convergence of the ADMM in Decentralized Consensus Optimization
Shi, W., Ling, Q., Yuan, K., Wu, G., & Yin, W. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2304432 · Cited by 820
Self-citationOn the Convergence of Decentralized Gradient Descent
Yuan, K., Ling, Q., & Yin, W. (2013) · arXiv · doi:10.48550/arxiv.1310.7063 · Cited by 635
Self-citationCommunication-Efficient Distributed Optimization using an Approximate Newton-type Method
Shamir, O., Srebro, N., & Zhang, T. (2013) · arXiv · doi:10.48550/arxiv.1312.7853 · Cited by 424
Xu, J., Zhu, S., Soh, Y. C., & Xie, L. (2015) · 2015 54th IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2015.7402509 · Cited by 332 · cited 1×
“Some recent works include techniques such as ADMM [16], [17], DLM [18], EXTRA [19], ESOM [20], DIGing [21], Aug-DGM [22] and exact diffusion [23], [24].”
MethodA Proximal Gradient Algorithm for Decentralized Composite Optimization
Shi, W., Ling, Q., Wu, G., & Yin, W. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2461520 · Cited by 307
Cited by
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Xin, R., Kar, S., & Khan, U. A. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2974267 · Cited by 114
“The nodes cooperate to solve the following problem: the regularization parameter is / , N 1 m = and the features are normalized to unit vectors [6], [38].”
BackgroundSupervised Learning Under Distributed Features
Ying, B., Yuan, K., & Sayed, A. H. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2881661 · Cited by 50
“In the simulation, we compare this implementation it with SAGA+Exact diffusion as proposed in [60].”
MethodSelf-citation
Metadata
- Authors
- Yuan, K., Ying, B., Liu, J., & Sayed, A. H.
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2019-01-15
- Volume
- 67
- Issue
- 2
- Pages
- 351-366
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2806414180
- Cited by
- 67
- References
- 9
- FWCI
- 6.54
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Variance-Reduced Stochastic Learning by Networked Agents Under Random Reshuffling. https://app.cochat.ai/papers/10.1109/tsp.2018.2872003
- The paper
- Yuan, K., Ying, B., Liu, J., & Sayed, A. H. (2019). Variance-Reduced Stochastic Learning by Networked Agents Under Random Reshuffling. IEEE Transactions on Signal Processing. https://doi.org/10.1109/tsp.2018.2872003
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.