The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks
Open accessAbstract
We propose an adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes. The cost function is assumed to consist of a collection of individual components. Diffusion adaptation allows the nodes to cooperate and diffuse information in real-time; it also helps alleviate the effects of stochastic gradient noise and measurement noise through a continuous learning process. We analyze the mean-square-error performance of the algorithm in some detail, including its transient and steady-state behavior. We also apply the diffusion algorithm to two problems: distributed estimation with sparse parameters and distributed localization. Compared to well-studied incremental methods, diffusion methods do not require the use of a cyclic path over the nodes and are robust to node and link failure. Diffusion methods also endow networks with adaptation abilities that enable the individual nodes to continue learning even when the cost function changes with time. Examples involving such dynamic cost functions with moving targets are common in the context of biological networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
Consensus Problems in Networks of Agents With Switching Topology and Time-Delays
Olfati-Saber, R., & Murray, R. M. (2004) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2004.834113 · Cited by 12,788
Reducibility among Combinatorial Problems
Karp, R. M. (1972) · Complexity of Computer Computations · doi:10.1007/978-1-4684-2001-2_9 · Cited by 11,511
Rogers, S. (1996) · Control Engineering Practice · doi:10.1016/0967-0661(96)82838-3 · Cited by 9,278
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Enhancing Sparsity by Reweighted ℓ 1 Minimization
Candès, E. J., Wakin, M. B., & Boyd, S. P. (2008) · Journal of Fourier Analysis and Applications · doi:10.1007/s00041-008-9045-x · Cited by 4,863
Compressive Sensing [Lecture Notes]
Baraniuk, R. (2007) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.4286571 · Cited by 4,177
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Regression Shrinkage and Selection via The Lasso: A Retrospective
Tibshirani, R. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2011.00771.x · Cited by 3,589
Distributed asynchronous deterministic and stochastic gradient optimization algorithms
Tsitsiklis, J., Bertsekas, D., & Athans, M. (1986) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1986.1104412 · Cited by 1,967
Adaptive filter theory (2nd ed.)
Haykin, S. (1991) · Prentice-Hall, Inc eBooks · Cited by 1,892
(1989) · Choice Reviews Online · doi:10.5860/choice.26-5109 · Cited by 1,755
Sayed, A. H., Tarighat, A., & Khajehnouri, N. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1458275 · Cited by 1,312
Self-citationDiffusion LMS Strategies for Distributed Estimation
Cattivelli, F. S., & Sayed, A. H. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2033729 · Cited by 1,174
Self-citationA scheme for robust distributed sensor fusion based on average consensus
Xiao, L., Boyd, S., & Lall, S. (2005) · doi:10.5555/1147685.1147698 · Cited by 1,170
Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis
Lopes, C. G., & Sayed, A. H. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.917383 · Cited by 1,109
Self-citationMatrix Analysis for Scientists and Engineers
Laub, A. J. (2004) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898717907 · Cited by 893
Sayed, A. H. (2008) · Wiley eBooks · doi:10.1002/9780470374122 · Cited by 848
Self-citationDiffusion Strategies for Distributed Kalman Filtering and Smoothing
Cattivelli, F. S., & Sayed, A. H. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2042987 · Cited by 842
Self-citationDistributed Stochastic Subgradient Projection Algorithms for Convex Optimization
Sundhar Ram, S., Nedić, A., & Veeravalli, V. V. (2010) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-010-9737-7 · Cited by 814 · cited 4×
“We can also set S = I in ATC and CTA to derive simplified versions that have no gradient exchange [29].”
MethodInfluentialGossip Algorithms for Distributed Signal Processing
Dimakis, A. G., Kar, S., Moura, J. M. F., Rabbat, M. G., & Scaglione, A. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2052531 · Cited by 741
Incremental Adaptive Strategies Over Distributed Networks
Lopes, C. G., & Sayed, A. H. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.896034 · Cited by 732
Self-citationA scheme for robust distributed sensor fusion based on average consensus
Xiao, L., Boyd, S., & Lall, S. (2005) · IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005. · doi:10.1109/ipsn.2005.1440896 · Cited by 713
Gossip Algorithms for Distributed Signal Processing
Dimakis, A. G., Kar, S., Moura, J. M. F., Rabbat, M., & Scaglione, A. (2018) · doi:10.1184/r1/6468842.v1 · Cited by 692
Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals
Schizas, I. D., Ribeiro, A., & Giannakis, G. B. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.906734 · Cited by 685
Diffusion recursive least-squares for distributed estimation over adaptive networks
Cattivelli, F. S., Lopes, C. G., & Sayed, A. H. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.913164 · Cited by 623
Self-citationIncremental Subgradient Methods for Nondifferentiable Optimization
Nedic, A., & Bertsekas, D. P. (2001) · SIAM Journal on Optimization · doi:10.1137/s1052623499362111 · Cited by 589
Distributed Sparse Linear Regression
Mateos, G., Bazerque, J. A., & Giannakis, G. B. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2055862 · Cited by 550
The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
Borkar, V. S., & Meyn, S. P. (2000) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012997331639 · Cited by 499
Broadcast Gossip Algorithms for Consensus
Aysal, T. C., Yildiz, M. E., Sarwate, A. D., & Scaglione, A. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2016247 · Cited by 480
Gradient Convergence in Gradient methods with Errors
Bertsekas, D. P., & Tsitsiklis, J. N. (2000) · SIAM Journal on Optimization · doi:10.1137/s1052623497331063 · Cited by 437
Quantized incremental algorithms for distributed optimization
Rabbat, M. G., & Nowak, R. D. (2005) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2005.843546 · Cited by 408
Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey
Bertsekas, D. P. (2011) · Optimization for Machine Learning · doi:10.7551/mitpress/8996.003.0006 · Cited by 381
Distributed Asynchronous Constrained Stochastic Optimization
Srivastava, K., & Nedic, A. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2118740 · Cited by 350
A New Class of Incremental Gradient Methods for Least Squares Problems
Bertsekas, D. P. (1997) · SIAM Journal on Optimization · doi:10.1137/s1052623495287022 · Cited by 312
Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis
Takahashi, N., Yamada, I., & Sayed, A. H. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2051429 · Cited by 310
Self-citationSensor Networks With Random Links: Topology Design for Distributed Consensus
Kar, S., & Moura, J. M. F. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.920143 · Cited by 261
Convergence and asymptotic agreement in distributed decision problems
Tsitsiklis, J., & Athans, M. (1984) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1984.1103385 · Cited by 247
Kar, S., & Moura, J. M. F. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2127446 · Cited by 231
Adaptive Learning in a World of Projections
Theodoridis, S., Slavakis, K., & Yamada, I. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.938752 · Cited by 217
Cooperative distributed multi-agent optimization
Nedić, A., & Ozdaglar, A. (2009) · Convex Optimization in Signal Processing and Communications · doi:10.1017/cbo9780511804458.011 · Cited by 185
Tu, S.-Y., & Sayed, A. H. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2125943 · Cited by 182
Self-citationDecentralized Parameter Estimation by Consensus Based Stochastic Approximation
Stankovic, S. S., Stanković, M. S., & Stipanović, D. M. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2076530 · Cited by 167
Adaptive Robust Distributed Learning in Diffusion Sensor Networks
Chouvardas, S., Slavakis, K., & Theodoridis, S. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2161474 · Cited by 165
Kopsinis, Y., Slavakis, K., & Theodoridis, S. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2090874 · Cited by 164
Modeling Bird Flight Formations Using Diffusion Adaptation
Cattivelli, F. S., & Sayed, A. H. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2107907 · Cited by 156
Self-citationBarbarossa, S., & Scutari, G. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.893921 · Cited by 147
Bio-Inspired Sensor Network Design
Barbarossa, S., & Scutari, G. (2007) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.361599 · Cited by 134
Adaptive Processing over Distributed Networks
Sayed, A. H., & Lopes, C. G. (2007) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1093/ietfec/e90-a.8.1504 · Cited by 131
Self-citationScutari, G., Barbarossa, S., & Pescosolido, L. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.909377 · Cited by 112
New Algorithms for Improved Adaptive Convex Combination of LMS Transversal Filters
Arenas‐García, J., Gómez-Verdejo, V., & Figueiras-Vidal, A. R. (2005) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2005.858823 · Cited by 109
Diffusion Least-Mean Squares Over Adaptive Networks
Lopes, C. G., & Sayed, A. H. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366830 · Cited by 108
Self-citationPlant identification via adaptive combination of transversal filters
Arenas‐García, J., Martínez‐Ramón, M., Navia-Vázquez, A., & Figueiras-Vidal, A. R. (2005) · Signal Processing · doi:10.1016/j.sigpro.2005.11.008 · Cited by 101
Diffusion strategies for distributed Kalman filtering: formulation and performance analysis
Cattivelli, F. S., Lopes, C. G., & Sayed, A. H. (2008) · Cited by 92
Self-citationDistributed processing over adaptive networks
Sayed, A. H., & Lopes, C. G. (2007) · International Symposium on Signal Processing and Its Applications (ISSPA) · doi:10.1109/isspa.2007.4555636 · Cited by 90
Self-citationFast Distributed Average Consensus Algorithms Based on Advection-Diffusion Processes
Sardellitti, S., Giona, M., & Barbarossa, S. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2032030 · Cited by 74
A diffusion rls scheme for distributed estimation over adaptive networks
Cattivelli, F. S., Lopes, C. G., & Sayed, A. H. (2007) · IEEE Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2007.4401393 · Cited by 56
Self-citationOptimal Distributed Online Prediction
Dekel, O., Gilad-Bachrach, R., Shamir, O., & Xiao, L. (2011) · Cited by 53
Modeling bee swarming behavior through diffusion adaptation with asymmetric information sharing
Li, J., & Sayed, A. H. (2012) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/1687-6180-2012-18 · Cited by 47
Self-citationDiffusion LMS algorithms with information exchange
Cattivelli, F. S., & Sayed, A. H. (2008) · 2018 52nd Asilomar Conference on Signals, Systems, and Computers · doi:10.1109/acssc.2008.5074402 · Cited by 38
Self-citationDiffusion Mechanisms For Fixed-Point Distributed Kalman Smoothing
Cattivelli, F. S., & Sayed, A. H. (2008) · doi:10.5281/zenodo.41205 · Cited by 33
Self-citationScalable distributed Kalman filtering through consensus
Kirti, S., & Scaglione, A. (2008) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2008.4518212 · Cited by 27
Bianchi, P., Fort, G., Hachem, W., & Jakubowicz, J. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947170 · Cited by 19
Distributed pareto-optimal solutions via diffusion adaptation
Chen, J., & Sayed, A. H. (2012) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2012.6319784 · Cited by 18
Self-citationDistributed optimization via diffusion adaptation
Chen, J., Tu, S.-Y., & Sayed, A. H. (2011) · IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing · doi:10.1109/camsap.2011.6136004 · Cited by 15
Self-citationDiffusion networks outperform consensus networks
Tu, S.-Y., & Sayed, A. H. (2012) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2012.6319691 · Cited by 12
Self-citationPerformance of diffusion adaptation for collaborative optimization
Chen, J., & Sayed, A. H. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288733 · Cited by 5
Self-citationData-Distributed Weighted Majority and Online Mirror DescentPreprint
Ouyang, H. J., & Gray, A. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1105.2274 · Cited by 3
Network optimization with heuristic rational agents
Eksin, C., & Ribeiro, A. (2011) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2011.6189953 · Cited by 1
Cited by
Distributed Optimization Over Time-Varying Directed Graphs
Nedic, A., & Olshevsky, A. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2364096 · Cited by 1,187
Speeding Up Distributed Machine Learning Using Codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2736066 · Cited by 829
On the Convergence of Decentralized Gradient Descent
Yuan, K., Ling, Q., & Yin, W. (2013) · arXiv · doi:10.48550/arxiv.1310.7063 · Cited by 635
On the Convergence of Decentralized Gradient Descent
Yuan, K., Ling, Q., & Yin, W. (2016) · SIAM Journal on Optimization · doi:10.1137/130943170 · Cited by 633
Fast Distributed Gradient Methods
Jakovetic, D., Xavier, J., & Moura, J. M. F. (2014) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2014.2298712 · Cited by 624
Metadata
- Authors
- Chen, J., & Sayed, A. H.
- Venue
- IEEE Transactions on Signal Processing
- Published
- 2012-08
- Volume
- 60
- Issue
- 8
- Pages
- 4289-4305
- ISSN
- 1053-587X (ISSN-L)
- 1941-0476
- OpenAlex
- W2029080014
- Cited by
- 680
- References
- 71
- FWCI
- 50.83
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.