A Proximal Gradient Algorithm for Decentralized Composite Optimization
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
This paper proposes a decentralized algorithm for solving a consensus optimization problem defined in a static networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form. Examples of such problems include decentralized constrained quadratic programming and compressed sensing problems, as well as many regularization problems arising in inverse problems, signal processing, and machine learning, which have decentralized applications. This paper addresses the need for efficient decentralized algorithms that take advantages of proximal operations for the nonsmooth terms. We propose a proximal gradient exact first-order algorithm (PG-EXTRA) that utilizes the composite structure and has the best known convergence rate. It is a nontrivial extension to the recent algorithm EXTRA. At each iteration, each agent locally computes a gradient of the smooth part of its objective and a proximal map of the nonsmooth part, as well as exchanges information with its neighbors. The algorithm is “exact” in the sense that an exact consensus minimizer can be obtained with a fixed step size, whereas most previous methods must use diminishing step sizes. When the smooth part has Lipschitz gradients, PG-EXTRA has an ergodic convergence rate of O(1/k) in terms of the first-order optimality residual. When the smooth part vanishes, PG-EXTRA reduces to P-EXTRA, an algorithm without the gradients (so no “G” in the name), which has a slightly improved convergence rate at o(1/k) in a standard (non-ergodic) sense. Numerical experiments demonstrate effectiveness of PG-EXTRA and validate our convergence results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“EXTRA converges at an o (1 /k ) rate when the objective function in (1) is convex, and it has a Q-linear rate when the objective function is strongly convex.”
From Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs (2017) · cites this paper as [61]
Citations
Cites
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719 · cited 8×
“We have hand-op-timized this parameter and the step sizes for DADMM, DSM, and P-EXTRA.”
MethodInfluentialAn Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination
Cao, Y., Yu, W., Ren, W., & Chen, G. (2013) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2012.2219061 · Cited by 2,399
Distributed asynchronous deterministic and stochastic gradient optimization algorithms
Tsitsiklis, J., Bertsekas, D., & Athans, M. (1986) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1986.1104412 · Cited by 1,967
Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms
Bullo, F., Cortés, J., & Martínez, S. (2009) · Princeton University Press eBooks · doi:10.1515/9781400831470 · Cited by 1,509
PROBLEMS IN DECENTRALIZED DECISION MAKING AND COMPUTATION
Τσιτσικλής, Ι. (2014) · doi:10.12681/eadd/3778 · Cited by 1,278
EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
Shi, W., Ling, Q., Wu, G., & Yin, W. (2015) · SIAM Journal on Optimization · doi:10.1137/14096668x · Cited by 1,261 · cited 4×
“When the possibly-nondifferentiable term , we have in (2a) and (2b) and thus (4) In (3a) and (3b), we have and thus (5) The updates (4) and (5) are known as EXTRA, a recent algo-rithm for decentralized differentiable optimization [23].”
InfluentialSelf-citationOn the Linear Convergence of the ADMM in Decentralized Consensus Optimization
Shi, W., Ling, Q., Yuan, K., Wu, G., & Yin, W. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2304432 · Cited by 820 · cited 1×
“The existing algorithms that solve problem (1) include the primal-dual domain methods such as the decentralized alternating direction method of multipliers (DADMM) [16], [21] and the primal domain methods including the distributed sub-gradient method (DSM) [22].”
MethodSelf-citationDistributed Stochastic Subgradient Projection Algorithms for Convex Optimization
Sundhar Ram, S., Nedić, A., & Veeravalli, V. V. (2010) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-010-9737-7 · Cited by 814
Fastest Mixing Markov Chain on a Graph
Boyd, S., Diaconis, P., & Xiao, L. (2004) · SIAM Review · doi:10.1137/s0036144503423264 · Cited by 744 · cited 1×
“Typical choices of can be found in [23], [28].”
BackgroundConsensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals
Schizas, I. D., Ribeiro, A., & Giannakis, G. B. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.906734 · Cited by 685 · cited 14×
“9 k 0 100 200 300 400 500 R el a ti v e E rr o r 10-10 10-8 10-6 10-4 10-2 100 DSM, step size ,k = 9:5=k1=2 DADMM, penalty coe/cient c = 0:0038 P-EXTRA, step size , = 46 k 0 100 200 300 400 500 S u cc es si ve D i, er en ce 10-10 10-5 100 105 DSM, step size ,k = 9:5=k1=2 DADMM, penalty coe/cient c = 0:0038 P-EXTRA, st…”
BackgroundMethodInfluential
Research works printed but not linked: 3
- davis, 2014, ?Convergence rates of relaxed Peaceman-Rachford and ADMM under regularity assumptions ?
- deng, 2013, ?Parallel multi-block ADMM with convergence ?
- 2011, Foundations of Location Analysis
Cited by
Items 101–200 of 368
Accelerated Distributed Dual Averaging Over Evolving Networks of Growing Connectivity
Liu, S., Chen, P., & Hero, A. O. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2793878 · Cited by 21
“In addition to distributed averaging, estimation and filter-ing, many distributed algorithms have been developed to solve general optimization problems [10]–[14], [26]–[33].”
BackgroundMethodInfluentialA Unifying Approximate Method of Multipliers for Distributed Composite Optimization
Wu, X., & Lu, J. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3173171 · Cited by 21
“The simulation involves PG-EXTRA [12], D-FBBS [13], DPGA [11], and the distributed ADMM [9], which are guaranteed to solve (52) with distinct h i ’s.”
BackgroundMethodInfluentialFast Attack-Resilient Distributed State Estimator for Cyber-Physical Systems
Yu, F., Dutta, R. G., Zhang, T., Hu, Y., & Jin, Y. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3013072 · Cited by 21
“The EXTRA algorithm required the cost function to be differentiable, and PG-EXTRA [25] was the non-differentiable version of the algorithm.”
MethodLiu, W., Mao, X., & Zhang, X. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3146785 · Cited by 21
“Furthermore, the authors of [34, 23] proposed the decentralized proximal gradient descent algorithms to bypass introducing the Lagrangian dual.”
BackgroundWang, Y., Liu, S., Sun, B. N., & Li, X. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3143157 · Cited by 20
“2) PG-EXTRA [17]: PG-EXTRA is proposed for composite problems with both of f and h existing, which enjoys an ergodic convergence rate O ( 1 k ) and reads As for our proposed DPPD, with the help of forward– backward splitting technique, (12) can be rewritten as Further, by substituting (11) as the…”
BackgroundMethodInfluentialA Geometrically Converging Dual Method for Distributed Optimization Over Time-Varying Graphs
Maros, M., & Jaldén, J. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3018743 · Cited by 21
“Distributed dual and primal-dual methods typically converge linearly to the optimal point using a constant step-size [14]–[16], [21]–[23],”
BackgroundMethodInfluentialCommunication Efficient Framework for Decentralized Machine Learning
Elgabli, A., Park, J., Bedi, A. S., Bennis, M., & Aggarwal, V. (2020) · Conference on Information Sciences and Systems · doi:10.1109/ciss48834.2020.1570627384 · Cited by 20
“There are a variety of distributed optimization algorithms proposed in the literature, such as primal methods [9], [10], [11], [12] and primal-dual methods [13], [14], [15].”
MethodDecentralized Consensus Optimization and Resource Allocation
Nedić, A., Olshevsky, A., & Shi, W. (2018) · Lecture notes in mathematics · doi:10.1007/978-3-319-97478-1_10 · Cited by 19
Self-citationWei, Y., Shang, C., Fang, H., Zeng, X., Dou, L., & Pardalos, P. M. (2021) · Automatica · doi:10.1016/j.automatica.2021.110071 · Cited by 19
Asynchronous Decentralized Accelerated Stochastic Gradient Descent
Lan, G., & Zhou, Y. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3080256 · Cited by 19
“, [11], [12], [13], [14], [15], [16], [17], [18], [19].”
Frugal and decentralised resolvent splittings defined by nonexpansive operators
Tam, M. K. (2023) · Optimization Letters · doi:10.1007/s11590-023-02064-y · Cited by 18
Decentralization Meets Quantization.
Tang, H., Zhang, C., Gan, S., Zhang, T., & Liu, J. (2018) · Cited by 18
A Distributed Nesterov-Like Gradient Tracking Algorithm for Composite Constrained Optimization
Zheng, L., Li, H., Li, J. Q., Wang, Z., Lü, Q., Shi, Y., Wang, H., Dong, T., Ji, L., & Xia, D. (2023) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2023.3239698 · Cited by 17
“By modifying PG-EXTRA [27], NIDS [29] improve the convergence rate and the dependency on the information of the network structure.”
BackgroundMethodResultInfluentialA fast proximal gradient algorithm for decentralized composite optimization over directed networks
Zeng, J., He, T., & Wang, M. (2017) · Systems & Control Letters · doi:10.1016/j.sysconle.2017.07.005 · Cited by 17
“Different from DPGM, Prox-DGD and DISTA, PG-EXTRA as an extension of EXTRA [13] has two interlaced sequences of iterates, whereas the proximal-gradient method just inherits the sequence of iterates in the gradient method. http://dx.doi.org/10.1016/j.sysconle.2017.07.005 0167-6911/© 2017 Elsevier B.V.”
InfluentialA Distributed Stochastic Proximal-Gradient Algorithm for Composite Optimization
Niu, Y., Li, H., Wang, Z., Lü, Q., Xia, D., & Ji, L. (2021) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2021.3065653 · Cited by 17
“The aforementioned algorithms [9]–[19] mainly focus on the smooth cases, and there also exists extensive work on distributed smooth + nonsmooth composite convex optimization problems [21]–[26], [43].”
BackgroundMethodInfluentialPMGT-VR: A Decentralized Proximal-Gradient Algorithmic Framework With Variance Reduction
Ye, H., Xiong, W., & Zhang, T. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3606874 · Cited by 17
“Decentralized Communication: In the decentralized setting, the absence of a parameter server requires agents to cooperatively minimize the function h ( x ) using only decentralized communications over a given network.”
Distributed Adaptive Gradient Algorithm With Gradient Tracking for Stochastic Nonconvex Optimization
Han, D., Liu, K., Lin, Y., & Xia, Y. (2024) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2024.3380710 · Cited by 16
“…and various algorithms have been developed over the past decade, e.g., distributed gradient descent [5], [6], [7], [8], distributed primal–dual gradient [9], [10], distributed dual averaging [11], [12], distributed mirror descent [13], [14], and distributed proximal gradient algorithms [15], [16].”
BackgroundA Second-Order Proximal Algorithm for Consensus Optimization
Wu, X., Qu, Z., & Lu, J. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.2996205 · Cited by 16
A distributed consensus-based optimal energy management approach in DC microgrids
Zhang, J., She, B., Peng, J. C., & Li, F. F. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108015 · Cited by 16
tvopt: A Python Framework for Time-Varying Optimization
Bastianello, N. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683695 · Cited by 15
Distributed hybrid optimization for multi-agent systems
Tan, X., Yang, Y., He, W., Cao, J., & Huang, T. (2022) · Science China Technological Sciences · doi:10.1007/s11431-022-2060-7 · Cited by 15
“After that, Shi et al. also extended EXTRA to solve the composite convex problems, and presented a proximal gradient EXTRA (PG-EXTRA) depending on the proximal operations [24].”
BackgroundLiu, C., Zhou, Z., Pei, J., Zhang, Y., & Shi, Y. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3209951 · Cited by 15
“1) Proximal gradient exact first-order algorithm (PGEXTRA) in [1] z = z(t−1) − x(t−1) + P̃(2x(t−1) − x(t−2)) − a(∇(t−1) −∇(t−2)) x = Proxh(z ) where P̃ = (I+P )⊗I 2 ,h(x) = ∑n i=1 h(xi), and Prox a h(z) := argminx∈Rmn{h(x) + 1 2a‖x− z‖2}.”
BackgroundMethodInfluentialHuang, Y., Meng, Z., Sun, J. Q., & Wang, G. (2023) · Automatica · doi:10.1016/j.automatica.2023.111309 · Cited by 15
Fast and privacy preserving distributed low-rank regression
Wai, H.-T., Scaglione, A., Lafond, J., & Moulines, É. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952998 · Cited by 15
Stochastic proximal splitting algorithm for composite minimization
Pătraşcu, A., & Irofti, P. (2021) · Optimization Letters · doi:10.1007/s11590-021-01702-7 · Cited by 15
Wang, J., & Hu, G. (2022) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6048 · Cited by 15
“(P5) min x ∑ i∈UC Ci(x i ) − ∑ j∈user Uj(x j ) subject to ∑ i∈UC xUC i = ∑ j∈user xuser j , (40) xUC i ∈ Ω UC i , ∀i ∈ UC, (41) xuser j ∈ Ω user j , ∀j ∈ user.”
Optimization Algorithm Design via Electric Circuits
Boyd, S., Parshakova, T., Ryu, E. K., & Suh, J. J. (2024) · neural information processing systems · doi:10.52202/079017-2172 · Cited by 14
Delay-Agnostic Asynchronous Distributed Optimization
Wu, X., Liu, C., Magnússon, S., & Johansson, M. (2023) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc49753.2023.10383647 · Cited by 14
“Distributed algorithms for solving Problem (1) include the distributed subgradient method [19], DGD [17], distributed gradient-tracking-based algorithm [20], distributed dual averaging [21], and PG-EXTRA [22].”
MethodInfluentialA Proximal ADMM for Decentralized Composite Optimization
Wang, B., Jiang, H., Fang, J., & Duan, H. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2841648 · Cited by 13
Distributed Optimization Over Networks
Nedić, A. (2018) · Lecture notes in mathematics · doi:10.1007/978-3-319-97142-1_1 · Cited by 13
D-FW: Communication efficient distributed algorithms for high-dimensional sparse optimization
Lafond, J., Wai, H.-T., & Moulines, É. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472457 · Cited by 13
Li, Z., & Yan, M. (2017) · arXiv (Cornell University) · Cited by 13
Decentralized Composite Optimization with CompressionPreprint
Li, Y., Liu, X., Tang, J., Yan, M., & Yuan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04448 · Cited by 13
“Decentralized algorithms such as PG-EXTRA (Shi et al., 2015b) and NIDS (Li et al., 2019) are proposed for composite optimization, and the sublinear rate is proved when the smooth component is strongly convex and smooth.”
BackgroundMethodPrimal-Dual Algorithm for Distributed Optimization with Coupled Constraints
Gong, K., & Zhang, L. (2024) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-024-02393-7 · Cited by 12
“Typical of such algorithms are EXTRA [40], PG-EXTRA [41], NIDS [18], ABC [48], and DCPA [5], etc.”
Distributed Momentum-Based Multiagent Optimization With Different Constraint Sets
Zhou, X., Ma, Z., Zou, S., & Margellos, K. (2024) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2024.3445575 · Cited by 12
“Reference [21] introduced a proximal-tracking distributed algorithm for this non-smooth problem with heterogeneous constraints by integrating the dynamic average consensus and adopting a constant step-size, which exhibited faster convergence than the P-EXTRA algorithm in [31].”
Zhou, Y., Guo, L., Shi, X., & Cao, J. (2024) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2024.3462519 · Cited by 12
“In fact, the proposed SDP-ADMM is closely related to the existing algorithms such as EXTRA [48], DLM [20], and PG-EXTRA [25].”
Latafat, P., & Patrinos, P. (2022) · Optimization methods & software · doi:10.1080/10556788.2021.2023524 · Cited by 12
“It is worth noting that finite sum minimization over graphs is another popular problem that has been considered by many authors [16, 20, 24, 27, 30, 37].”
BackgroundA One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite OptimizationPreprint
Xiao, T., Chen, X., Balasubramanian, K., & Ghadimi, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09766 · Cited by 12
“There exists a constant ν ≥ 0 such that for all”
BackgroundA first-order algorithm for decentralised min-max problems
Malitsky, Y., & Tam, M. K. (2026) · Numerische Mathematik · doi:10.1007/s00211-026-01530-w · Cited by 11
“For the purpose of this work, our discussion will focus on a method from the latter class known as PG-EXTRA [26].”
InfluentialDistributed (ATC) Gradient Descent for High Dimension Sparse Regression
Yao, J., Scutari, G., Sun, Y., & Honnappa, H. (2023) · IEEE Transactions on Information Theory · doi:10.1109/tit.2023.3267742 · Cited by 11
“…optimization is plenty of schemes but DGD; they differ from plain DGD for implementing some form of correction of the local gradient direction, via distributed tracking mechanisms of the full gradient [25], [28], [31], [39], [40], [48] or using dual variables [16], [17], [18], [35], [36], [37].”
MethodAsynchronous Distributed Optimization With Delay-free Parameters
Wu, X., Liu, C., Magnússon, S., & Johansson, M. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3590953 · Cited by 11
“…including distributed model predictive control [27], empirical risk minimization [28], etc. Distributedalgorithmsforsolvingproblem(1)includethedis-tributed subgradient method [29], Prox-DGD [24], distributed gradient-tracking-based algorithm [30], distributed dual averaging [31], and PG-EXTRA [32].”
Graph Simplification-Aided ADMM for Decentralized Composite Optimization
Wang, B., Fang, J., Duan, H., & Li, H. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2953538 · Cited by 11
“In some other works, e.g., [29]–[32], [35], and [36], a different consensus formation was employed by resorting to the Laplacian matrix of the underlying graph.”
MethodInfluentialDistributed Learning Systems with First-order Methods
Liu, J., & Zhang, C. (2020) · doi:10.1561/9781680837018 · Cited by 11
DN-ADMM: Distributed Newton ADMM for Multi-agent Optimization
Li, Y., Freris, N. M., Voulgaris, P. G., & Stipanović, D. M. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683317 · Cited by 10
A Unified Algorithmic Framework for Distributed Composite Optimization
Xu, J., Tian, Y., Sun, Y., & Scutari, G. (2020) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc42340.2020.9303791 · Cited by 10
“The above algorithm subsumes many existing proximal-gradient methods, such as PG-EXTRA [15] or ID-FBBS [16].”
BackgroundMethodA divide-and-conquer algorithm for distributed optimization on networks
Emirov, N., Song, G., & Sun, Q. (2024) · Applied and Computational Harmonic Analysis · doi:10.1016/j.acha.2023.101623 · Cited by 10
Hu, J., Xia, D., Cheng, H., Feng, L., Ji, L., Guo, J., & Li, H. (2020) · Asian Journal of Control · doi:10.1002/asjc.2483 · Cited by 10
“There have been some remarkable works, including ADMM [15], EXTRA [16], and PG-EXTRA [17].”
BackgroundResultLi, H., Ding, W., Wang, Z., Lü, Q., Ji, L., Li, Y., & Huang, T. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3144187 · Cited by 10
“Previous works on proximal splitting methods include some centralized algorithms [30]–[34] and decentralized proximal gradient algorithms [35]–[39], such as PG-ADMM in [37] and [38] for optimization problems with nonsmooth local objective functions.”
BackgroundMethodInfluentialCommunication Efficient Curvature Aided Primal-Dual Algorithms for Decentralized Optimization
Li, Y., Voulgaris, P. G., Stipanović, D. M., & Freris, N. M. (2023) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2023.3244904 · Cited by 10
“In this section, we present a comparative experimental validation of the proposed methods with existing state-of-theart methods, namely PGE [10], P2D2 [12], and ESOM [33].”
MethodA projection-free decentralized algorithm for non-convex optimization
Wai, H.-T., Scaglione, A., Lafond, J., & Moulines, É. (2016) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2016.7905887 · Cited by 10
“▶ requires convexity except for [Hon16].”
Accelerated Dual Averaging Methods for Decentralized Constrained Optimization
Liu, C., Shi, Y., Li, H., & Du, W. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3173062 · Cited by 9
“To improve the performance using a constant step size, a variant of EXTRA (PG-EXTRA) was developed in [21], where the constraint is modeled as a non-smooth indicator function and handled via the proximal operator.”
Regularized Diffusion Adaptation via Conjugate Smoothing
Vlaski, S., Vandenberghe, L., & Sayed, A. H. (2021) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2021.3081073 · Cited by 9
On the Linear Convergence of Two Decentralized Algorithms
Li, Y., & Yan, M. (2021) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-021-01833-y · Cited by 9
Argus: Federated Non-Convex Bilevel Learning Over 6G Space-Air-Ground Integrated Network
Liu, Y., Yang, K., Zhu, Y., Yang, K., & Zhao, H. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3582695 · Cited by 8
“Inspired by [57], to cope with the non-smooth term, we use a variant of (3) as follows: Thenwecanuse K communicationroundstoapproximate y ∗ .”
BFGS-ADMM for Large-Scale Distributed Optimization
Li, Y., Gong, Y., Freris, N. M., Voulgaris, P. G., & Stipanović, D. M. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683678 · Cited by 8
“…In this section we present some numerical experiments of the proposed method compared to distributed first order methods: P2D2 [22] and PG-EXTRA [7], for solving the following distributed logistic regression problem: where f i ( x ) = 1 m i m i j =1 ln 1 + e w (cid:62) j x + (1 − y j ) w j x ,…”
MethodDistributed Composite Optimization Over Relay-Assisted Networks
Shi, C., & Yang, G. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2963452 · Cited by 8
“The considered algorithms include the algorithm (9a)–(9d), the PGC [12], the PG-EXTRA [37], and the DSG [30].”
BackgroundMethodInfluentialA Communication Efficient Quasi-Newton Method for Large-Scale Distributed Multi-Agent Optimization
Li, Y., Voulgaris, P. G., & Freris, N. M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747099 · Cited by 8
“EXPERIMENTS We evaluate the proposed L-BFGS−ADMM against existing distributed algorithms for multi-agent convex composite optimization problems, namely P2D2 [7] and PG-EXTRA [6].”
MethodDecentralized Statistical Inference with Unrolled Graph Neural Networks
Wang, H., Shen, Y., Wang, Z., Li, D., Zhang, J., Letaief, K. B., & Lu, J. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9682857 · Cited by 8
Komuro, K., Yukawa, M., & Cavalcante, R. L. G. (2022) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2022.3181729 · Cited by 8
A Decentralized Stochastic Algorithm for Coupled Composite Optimization With Linear Convergence
Lü, Q., Liao, X., Deng, S., & Li, H. (2022) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2022.3190743 · Cited by 8
Distributed Algorithms for Solving Locally Coupled Optimization Problems on Agent Networks
Hu, J., Xiao, Y., & Liu, J. (2018) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2018.8619467 · Cited by 8
“, subgradient-based methods [5], [6], dual-decomposition methods [7], proximal gradient descent methods [8], and general first order algorithms [9], [10], [11], to name a few.”
MethodWei, X., Yu, H., Ling, Q. H., & Neely, M. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.01275 · Cited by 8
“This result improves upon the performance bounds of the previously known decentralized optimization algorithms (e.g. PG-EXTRA Shi et al. (2015) and decentralized ADMM Shi et al. (2014)), which do not take into account the special structure of the problem and only obtain a convergence time of O (1 /ε ) .”
BackgroundMethodResultInfluentialSelf-citationMaros, M., Scutari, G., Sun, Y., & Cheng, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.08507 · Cited by 8
“Examples include: gradient tracking methods [34, 28, 47, 12, 43] and primal-dual schemes [17, 41, 16, 40, 42, 24], just to name a few.”
Guo, L., Wang, L., Shi, X., & Cao, J. (2025) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2025.3600766 · Cited by 7
“Various algorithms have been proposed to solve problem (1) in a decentralized manner, such as EXTRA [4], [5], DIGing [7], GT [6], NEXT [8], AccNEXT [9], NIDS [10] (or ED [11]), SONATA [12], D-iPGM [13], DISA [14], and VR-DPPD [15].”
MethodDistributed Optimization Under Information Constraints: A Survey
Liu, S., Hua, Y., Han, Q., & Xie, L. (2026) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2025.3645932 · Cited by 7
“For instance, PG-EXTRA was developed by incorporating proximal operators [11].”
Decentralized Douglas-Rachford Splitting Methods for Smooth Optimization over Compact Submanifolds
Deng, K., Hu, J., & Wang, H. (2024) · Journal of Computational Mathematics · doi:10.4208/jcm.2407-m2023-0282 · Cited by 7
Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine
Chen, C., Qiao, N., & Zhu, L. (2025) · Journal of the American Statistical Association · doi:10.1080/01621459.2025.2550671 · Cited by 7
“Further works by Shi et al. (2015) and Li, Shi, and Yan (2019) developed decentralized (proximal) gradient descent algorithms that avoid the use of Lagrangian duals.”
A Parameter-free Decentralized Algorithm for Composite Convex Optimization
Chen, X., Kuruzov, I. A., Scutari, G., & Gasnikov, A. V. (2025) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc57313.2025.11312133 · Cited by 7
“In this section we introduce a variant of Algorithm II-B wherein the global min-consensus step (S. Notice that Algorithm 3 differs from Algorithm 2 also in the updates of the D -variables, due to the fact that Λ k / ∈ span ( I ) , for all k .”
GradConsensus: Linearly Convergent Algorithm for Reducing Disagreement in Multi-Agent Optimization
Khatana, V., Saraswat, G., Patel, S., & Salapaka, M. V. (2023) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2023.3321757 · Cited by 7
“However, most existing works including, [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], impose restrictive conditions such as the requirement of doubly-stochastic weight matrices and need of balanced undirected graphs.”
InfluentialShen, J., Wang, L., & Zhu, L. (2023) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2023.109669 · Cited by 7
Bridging the gap between big data and game theory: A general hierarchical pricing framework
Zheng, Z., Song, L., & Han, Z. (2017) · International Conference on Communications · doi:10.1109/icc.2017.7996334 · Cited by 7
Decentralized constrained consensus optimization with primal dual splitting projection
Zhang, H., Shi, W., Mokhtari, A., Ribeiro, A., & Ling, Q. H. (2016) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2016.7905905 · Cited by 7
Self-citationDynamic Average Diffusion With Randomized Coordinate Updates
Ying, B., Yuan, K., & Sayed, A. H. (2019) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2019.2942191 · Cited by 7
“For simplicity, we shall assume in this work that each Jk(w) is differentiable although the analysis can be extended to non-smooth risk functions by employing subgradient constructions, along the lines of [7], [54], or proximal constructions similar to [5], [55].”
MethodGradient-Consensus: Linearly Convergent Distributed Optimization Algorithm over Directed GraphsPreprint
Khatana, V., Saraswat, G., Patel, S., & Salapaka, M. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10070 · Cited by 7
“The works in [12], [13] have developed algorithms based on proximal-gradient to tackle (1) with proximal friendly fi’s.”
BackgroundAccelerated Infeasibility Detection of Constrained Optimization and Fixed-Point IterationsPreprint
Park, J., & Ryu, E. K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.15876 · Cited by 7
“This SDP can be solved in decentralized manner with PG-EXTRA of Shi et al. (2015).”
MethodDecentralized Proximal Method of Multipliers for Convex Optimization with Coupled ConstraintsPreprint
Gong, K., & Zhang, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.15596 · Cited by 7
“√ Reference [16] uses U = (cid:32)L, while we use the decomposition (cid:32)L = U ⊤ U .”
MethodInfluentialConvergence Analysis of EXTRA in Non-Convex Distributed Optimization
Qin, L., & Pu, Y. (2025) · IEEE Control Systems Letters · doi:10.1109/lcsys.2025.3577964 · Cited by 6
“Additionally, convergence results for EXTRA on directed graphs were established in [19], while extensions such as PG-EXTRA [20] have addressed non-smooth convex problems.”
Logarithmic Communication for Distributed Optimization in Multi-Agent Systems
London, P., Vardi, S., & Wierman, A. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366696 · Cited by 6
“, primal decomposition [9, 45], dual decomposition [13, 37, 40, 50, 60, 83, 86], subgradient methods [59, 62], and proximal gradient descent methods [81], to name a few.”
MethodMulti-Model Method Decentralized Adaptive Control for a Class of Discrete-Time Multi-Agent Systems
Zhang, X., Chang, Z., & Wang, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3030635 · Cited by 6
“…system independently faces to unknown parameters, the researchers have presented many estimation algorithms such as the gradient algorithm [18], the maximum likelihood algorithm [19], the least-squares algorithm [20], the backstepping approach [21], the observer method and sliding mode…”
MethodWang, J., & Hu, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.00956 · Cited by 6
Wang, L., Bao, L., & Liu, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.11573 · Cited by 6
“To the best of our knowledge, this achieves the best result in the literature.”
Augmented distributed optimization for networked systems
Xu, J. (2019) · doi:10.32657/10356/68805 · Cited by 5
Golden Ratio Proximal Gradient ADMM for Distributed Composite Convex Optimization
Yin, C., & Yang, J. (2023) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-023-02336-8 · Cited by 5
“In [38], the authors proposed a proximal gradient exact first-order algorithm (PGEXTRA), which extends EXTRA to handle the non-smooth functions { ξ i } i ∈ N .”
InfluentialDistributed Proximal Algorithms for Nonsmooth Optimization: Unified Convergence Analysis
Huang, Y., Cui, S., Sun, J. Q., & Meng, Z. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3576271 · Cited by 5
“By virtue of the attractive property, some distributed proximal algorithms (e.g., PG-EXTRA and PG-ADMM) were proposed in [17], [18], [19], [20], and [21].”
Globally Constrained Decentralized Optimization With Variable Coupling
Wang, D., Wu, X., Ou, Z., & Lu, J. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3629001 · Cited by 5
“We first make an attempt to solve the dual problem (13) by employing the distributed first-order method P-EXTRA [42].”
Liu, J., Wang, Z., & Wang, Y. (2026) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2026.3733955 · Cited by 5
“In addition, we make the following standard assumption [40], [42], [44] on the communication network G = {V , E} .”
AdGT: Decentralized Gradient Tracking With Adaptive Per-Agent Stepsizes
Ghaderyan, D., & Werner, S. (2026) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2026.3733814 · Cited by 5
“In addition, the work in [9] investigated a proximal-gradient method for decentralized optimization.”
Inexact proximal gradient algorithm with random reshuffling for nonsmooth optimization
Jiang, X., Fang, Y., Zeng, X., Sun, J. Q., & Chen, J. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-4095-y · Cited by 5
“For achieving nonsmooth convex optimization (1), one efficient method is the proximal gradient al-gorithm, where the proximal operator is used as an approximate gradient for gradient descent [4–7].”
A Deep Reinforcement Learning Approach to Efficient Distributed Optimization
Zhu, D., Xu, T., & Lu, J. (2025) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2025.3587331 · Cited by 5
“4(b), we compare the learned policy with a hand-selected fixed policy (i.e., π ( a c | s ) ≡ 1 ), and, similarly as in Section V-A, incorporate PG-EXTRA [8], DPGA [10], D-FBBS [11], and PGC [9] into the comparison to demonstrate the competitive performance of our learning-based framework.”
InfluentialA forward-backward Bregman splitting scheme for regularized distributed optimization problems
Xu, J., Zhu, S., Soh, Y. C., & Xie, L. (2016) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2016.7798413 · Cited by 5
“Remark 4: The Bregman method and operator splitting technique used to develop the algorithm, indeed, provide a unified framework which allows us to easily recover most of the existing algorithm, such as EXTRA/P-EXTRA/PGEXTRA [7], [24], D-ADMM/DLM [18], [20] and CADMM/IC-ADMM [19], [32] by properly designing the consen…”
BackgroundMethodDecentralized bundle method for nonsmooth consensus optimization
Wang, Z., Ling, Q. H., & Yin, W. (2017) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2017.8309023 · Cited by 5
“Similar to DSM, DDA and PG-EXTRA, DBM can use a symmetric weight matrix W following the Laplacian-based constant Algorithm 1 Decentralized Bundle Method (DBM) at Agent i 1: Initialize constants δ̄ > 0, m ∈ (0, 1), µi > 0, {wij}, as well as variables x0i and p0i .”
BackgroundSelf-citationDecentralized proximal splitting algorithms for composite constrained convex optimization
Zheng, L., Ran, L., Li, H., Feng, L., Wang, Z., Lü, Q., & Xia, D. (2022) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2022.07.053 · Cited by 5
A Data-Driven Loss Weighting Scheme Across Heterogeneous Tasks for Image Denoising
Rui, X., Cao, X., Zhao, X.-L., Meng, D., & Ng, M. K. (2026) · SIAM Journal on Imaging Sciences · doi:10.1137/25m178403x · Cited by 5
Ren, J., & Haupt, J. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2996374 · Cited by 5
“In the synchronous setting, we compare with the algorithm PGEXTRA in [25].”
BackgroundMethodResultInfluentialAn adaptive finite-time neurodynamic approach to distributed consensus-based optimization problem
Li, Q., Wang, M., Sun, H., & Qin, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08794-5 · Cited by 5
“Different from most existing results in [32, 33], these results require each local objective function fiðxiÞ to be strongly convex.”
BackgroundResultDistributed Sparse Regression via PenalizationPreprint
Yao, J., Scutari, G., Sun, Y., & Honnappa, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06530 · Cited by 5
“…examples include: gradient tracking methods (Qu and Li, 2017, Nedić et al., 2016, Xu et al., 2018, Lorenzo and Scutari, 2016, Sun et al., 2019) and primal-dual schemes (Jakovetić et al., 2011, Shi et al., 2014, Jakovetić, 2019, Jakovetić et al., 2013, Shi et al., 2015a,b), just to name a few.”
MethodDistributed Normal Map-based Stochastic Proximal Gradient Methods over NetworksPreprint
Huang, K., Pu, S., & Nedić, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.13054 · Cited by 5
“To improve the algorithmic convergence, there are mainly two lines of works, including [30], [31] that apply the gradient tracking technique [30], [32], [33], and primal-dual type methods including [34]– [37] which are inspired by earlier works assuming φ = 0 [38], [39].”
Differentially Private Decentralized Optimization With Relay Communication
Wang, L., Guo, L., Yang, S., & Shi, X. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3515803 · Cited by 4
“Specifically, we assess DP-ADMM [12], DP-NIDS [28], DP-EXTRA [27], DP-RWADMM [30], and our proposed approach DP-RECAL.”
InfluentialXiong, M., Yang, C., Zhao, J., Zhang, B., & Yuan, D. (2025) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2025.3632693 · Cited by 4
“Currently, several studies on composite optimization have been published, with most focusing on offline distributed composite optimization (see, e.g., [16], [17], [18], [19], and [20]) and centralized online composite optimization (see, e.g., [21], [22], [23], and [24]), to name a few.”
Multi-consensus decentralized primal-dual fixed point algorithm for distributed learning
Tang, K., Liu, W., & Mao, X. (2024) · Machine Learning · doi:10.1007/s10994-024-06537-8 · Cited by 4
“This technique was studied in the literature, e.g., by Shi et al. (2015) and Qu and Li (2017).”
Influential
Metadata
- Authors
- Shi, W., Ling, Q., Wu, G., & Yin, W.
- Affiliations
- University of Science and Technology of China (CN)
- University of California, Los Angeles (US)
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2015-11
- Volume
- 63
- Issue
- 22
- Pages
- 6013-6023
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W1627780903
- Cited by
- 307
- References
- 36
- Citations by year
- 2026: 17 · 2025: 30 · 2024: 31 · 2023: 28 · 2022: 36 · 2021: 35 · 2020: 41 · 2019: 27 · 2018: 28 · 2017: 19 · 2016: 15
- FWCI
- 23.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Signal Processing (publishedVersion)
- Funders
- National Natural Science Foundation of China (61004137, 61573331)
- National Science Foundation (DMS-1317602)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Signal Processing › Sparse and Compressive Sensing Techniques
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Optimization and Variational Analysis
- Keywords
- distributed optimization, consensus optimization, multi-agent systems, proximal gradient method, composite objective function, nonsmooth optimization, fixed step size, ergodic convergence rate, compressed sensing, regularization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.