Stochastic proximal gradient consensus over time-varying networks
The DOI matches its Crossref record; 31 of the 33 references it cites were found in a registry or an index.
Abstract
We consider solving a convex, nonsmooth and stochastic optimization problem over a multi-agent network. Each agent has access to a local objective function and can communicate with its immediate neighbors only. We develop a dynamic stochastic proximal-gradient consensus (DySPGC) algorithm, featuring: i) it works for both the static and randomly time-varying networks; ii) it can deal with either the exact or the stochastic gradient information; iii) it has provable rate of convergence. Interestingly, the developed algorithm includes as special cases many existing (and seemingly unrelated) first-order algorithms for distributed optimization over static networks, such as the EXTRA (Shi et al 2014), the PG-EXTRA (Shi at 2015), the IC/IDC-ADMM (Chang et al 2014), and the DLM (Ling et al 2015). It is also closely related to the classical distributed gradient method.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In particular, ADMM over a random undirected network has been shown to have O (1 /k ) rate for convex functions [25].”
From Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs (2017) · cites this paper as [25]
Citations
Cites
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Parikh, N., & Boyd, S. (2014) · Foundations and Trends® in Optimization · doi:10.1561/2400000003 · Cited by 4,102
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Proximal Splitting Methods in Signal Processing
Combettes, P. L., & Pesquet, J.-C. (2011) · Springer Optimization and Its Applications · doi:10.1007/978-1-4419-9569-8_10 · Cited by 2,183
Constrained Consensus and Optimization in Multi-Agent Networks
Nedic, A., Ozdaglar, A., & Parrilo, P. A. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2041686 · Cited by 2,086
Numerical Methods for Nonlinear Variational Problems
Glowinski, R. (1984) · Springer eBooks · doi:10.1007/978-3-662-12613-4 · Cited by 1,728
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
Shi, W., Ling, Q., Wu, G., & Yin, W. (2015) · SIAM Journal on Optimization · doi:10.1137/14096668x · Cited by 1,261
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2012) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2011.2161027 · Cited by 947
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2010) · Cited by 884
Hong, M., Luo, Z.-Q., & Razaviyayn, M. (2016) · SIAM Journal on Optimization · doi:10.1137/140990309 · Cited by 841
Self-citationOn the Linear Convergence of the ADMM in Decentralized Consensus Optimization
Shi, W., Ling, Q., Yuan, K., Wu, G., & Yin, W. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2304432 · Cited by 820
Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization
Sundhar Ram, S., Nedić, A., & Veeravalli, V. V. (2010) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-010-9737-7 · Cited by 814 · cited 1×
“…〉 + hr+1(x) + 1 2 ∥∥Ar+1x+Br+1zr + Γ−1λr∥∥2 Γ + 1 2 ‖x− xr‖2Ωr+1+ηr+1IMN (7a) xr+1i = x r i , if i /∈ Vr+1 (7b) zr+1 = argmin 1 2 ∥∥Ar+1xr+1 +Br+1z + Γ−1λr∥∥2 Γ (7c) zr+1ij = z r ij , if eij /∈ Ar+1 (7d) λr+1 = λr + Γ ( Ar+1xr+1 +Br+1zr+1 ) (7e) When the network is static and the exact gradient…”
Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals
Schizas, I. D., Ribeiro, A., & Giannakis, G. B. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.906734 · Cited by 685
Fast Distributed Gradient Methods
Jakovetic, D., Xavier, J., & Moura, J. M. F. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2298712 · Cited by 624
Multi-Agent Distributed Optimization via Inexact Consensus ADMM
Chang, T.-H., Hong, M., & Wang, X. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2367458 · Cited by 444 · cited 5×
“Additionally our method generalizes other distributed ADMM-type methods such as the DLM [20] and the IC-ADMM [19].”
InfluentialSelf-citationD-ADMM: A Communication-Efficient Distributed Algorithm for Separable Optimization
Mota, J. F. C., Xavier, J. M. F., Aguiar, P. M. Q., & Puschel, M. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2254478 · Cited by 363
On the O(1=k) convergence of asynchronous distributed alternating Direction Method of Multipliers
Wei, E., & Ozdaglar, A. (2013) · 2013 IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2013.6736937 · Cited by 342
An Accelerated Linearized Alternating Direction Method of Multipliers
Ouyang, Y., Chen, Y., Lan, G., & Pasiliao, E. (2015) · SIAM Journal on Imaging Sciences · doi:10.1137/14095697x · Cited by 217
DLM: Decentralized Linearized Alternating Direction Method of Multipliers
Ling, Q., Shi, W., Wu, G., & Ribeiro, A. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2436358 · Cited by 196
Proximal Splitting Methods in Signal ProcessingPreprint
Combettes, P. L., & Pesquet, J.-C. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0912.3522 · Cited by 188
Distributed subgradient methods and quantization effects
Nedic, A., Olshevsky, A., Ozdaglar, A., & Tsitsiklis, J. N. (2008) · 2008 47th IEEE Conference on Decision and Control · doi:10.1109/cdc.2008.4738860 · Cited by 185
Hong, M., Luo, Z.-Q. T., & Razaviyayn, M. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178689 · Cited by 171
Self-citationFast Consensus by the Alternating Direction Multipliers Method
Erseghe, T., Zennaro, D., Dall'Anese, E., & Vangelista, L. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2162831 · Cited by 163
Some Saddle-function splitting methods for convex programming
Eckstein, J. (1994) · Optimization methods & software · doi:10.1080/10556789408805578 · Cited by 142
On the Information-Adaptive Variants of the ADMM: An Iteration Complexity Perspective
Gao, X., Jiang, B., & Zhang, S. (2017) · Journal of Scientific Computing · doi:10.1007/s10915-017-0621-6 · Cited by 122
Coding With Side Information for Rate-Constrained Consensus
Yildiz, M. E., & Scaglione, A. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.919636 · Cited by 77
Distributed consensus-based demodulation: algorithms and error analysis
Zhu, H., Caño, A., & Giannakis, G. B. (2010) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2010.06.090890 · Cited by 58
Fast Distributed First-Order Methods
Chen, A. I.-A. (2012) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 49
A proximal gradient algorithm for decentralized nondifferentiable optimization
Shi, W., Ling, Q. H., Wu, G., & Yin, W. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178514 · Cited by 10
Research works printed but not linked: 1
- hong, 2015, Stochastic proximal gradient consensus over time-varying networks, Technical Report
Cited by
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs
Nedić, A., Olshevsky, A., & Shi, W. (2017) · SIAM Journal on Optimization · doi:10.1137/16m1084316 · Cited by 1,069
Stochastic Proximal Gradient Consensus Over Random Networks
Hong, M., & Chang, T. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2673815 · Cited by 85
Self-citationHybrid Voltage Control in Distribution Networks Under Limited Communication Rates
Liu, H. J., Shi, W., & Zhu, H. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2797692 · Cited by 75
Lü, Q., Li, H., & Xia, D. (2017) · Information Sciences · doi:10.1016/j.ins.2017.09.038 · Cited by 56
Lü, Q., & Li, H. (2016) · arXiv · doi:10.48550/arxiv.1611.00990 · Cited by 54
Metadata
- Authors
- Hong, M., & Chang, T.-H.
- Affiliations
- Iowa State University (US)
- Chinese University of Hong Kong, Shenzhen (CN)
- Venue
- 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2016-03
- Pages
- 4776-4780
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2403721507
- Cited by
- 6
- References
- 33
- Citations by year
- 2021: 1 · 2018: 1 · 2017: 3 · 2016: 1
- FWCI
- 1.42
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Optimization and Variational Analysis
- Keywords
- consensus algorithms, multi-agent systems, dynamic networks, distributed optimization, nonsmooth convex optimization, stochastic gradient, convergence rate, alternating direction method of multipliers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.