Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior
Open accessThe DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.
Abstract
Nature provides splendid examples of real-time learning and adaptation behavior that emerges from highly localized interactions among agents of limited capabilities. For example, schools of fish are remarkably apt at configuring their topologies almost instantly in the face of danger [1]: when a predator arrives, the entire school opens up to let the predator through and then coalesces again into a moving body to continue its schooling behavior. Likewise, in bee swarms, only a small fraction of the agents (about 5%) are informed, and these informed agents are able to guide the entire swarm of bees to their new hive [2]. It is an extraordinary property of biological networks that sophisticated behavior is able to emerge from simple interactions among lower-level agents [3].
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Typical decentralized aggregation approaches include the consensusbased methods [69] and diffusion strategies [70].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [70]
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Coordination of groups of mobile autonomous agents using nearest neighbor rules
Jadbabaie, A., Lin, J., & Morse, A. S. (2003) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2003.812781 · Cited by 8,339
Flocks, herds and schools: A distributed behavioral model
Reynolds, C. W. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37406 · Cited by 7,714
Novel Type of Phase Transition in a System of Self-Driven Particles
Vicsek, T., Czirók, A., Ben-Jacob, E., Cohen, I., & Shochet, O. (1995) · Physical Review Letters · doi:10.1103/physrevlett.75.1226 · Cited by 7,380
Parallel and Distributed Computation: Numerical Methods
Bertsekas, D. P., & Tsitsiklis, J. N. (1989) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 5,293
Flocking for Multi-Agent Dynamic Systems: Algorithms and Theory
Olfati-Saber, R. (2006) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2005.864190 · Cited by 5,037
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Bellanger, M. (1987) · Signal Processing · doi:10.1016/0165-1684(87)90147-2 · Cited by 2,961
Fast linear iterations for distributed averaging
Xiao, L., & Boyd, S. (2004) · Systems & Control Letters · doi:10.1016/j.sysconle.2004.02.022 · Cited by 2,706
Jackson, M. O. (2008) · Princeton University Press eBooks · doi:10.1515/9781400833993 · Cited by 2,666
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,650
Alexander, S. T. (1986) · Springer eBooks · doi:10.1007/978-1-4612-4978-8 · Cited by 2,646
Collective Memory and Spatial Sorting in Animal Groups
COUZIN, I. D., KRAUSE, J., JAMES, R., RUXTON, G. D., & FRANKS, N. R. (2002) · Journal of Theoretical Biology · doi:10.1006/jtbi.2002.3065 · Cited by 2,138
Robust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
Stochastic Approximation and Recursive Algorithms and Applications
(2003) · Stochastic Modelling and Applied Probability · doi:10.1007/b97441 · Cited by 1,838
Diffusion LMS Strategies for Distributed Estimation
Cattivelli, F. S., & Sayed, A. H. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2009.2033729 · Cited by 1,174
Self-citationSelf-organization in biological systems
(2002) · Choice Reviews Online · doi:10.5860/choice.39-3374 · Cited by 1,124
Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis
Lopes, C. G., & Sayed, A. H. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.917383 · Cited by 1,109
Self-citationInferring the structure and dynamics of interactions in schooling fish
Katz, Y., Tunstrøm, K., Ioannou, C. C., Huepe, C., & Couzin, I. D. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1107583108 · Cited by 927
Sayed, A. H. (2008) · Wiley eBooks · doi:10.1002/9780470374122 · Cited by 848
Self-citationDiffusion Strategies for Distributed Kalman Filtering and Smoothing
Cattivelli, F. S., & Sayed, A. H. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2042987 · Cited by 842
Self-citationDistributed Stochastic Subgradient Projection Algorithms for Convex Optimization
Sundhar Ram, S., Nedić, A., & Veeravalli, V. V. (2010) · Journal of Optimization Theory and Applications · doi:10.1007/s10957-010-9737-7 · Cited by 814 · cited 2×
“The CTA structure of Figure 2, with the additional requirement that the step size n is a time-dependent sequence ( ) i n that decays toward zero with time as in (6), was later employed by [39]–[41] to solve distributed optimization problems that require all agents to reach agreement—see the section “Distributed Optimi…”
BackgroundMethodCooperative behavior cascades in human social networks
Fowler, J. H., & Christakis, N. A. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0913149107 · Cited by 767
Fastest Mixing Markov Chain on a Graph
Boyd, S., Diaconis, P., & Xiao, L. (2004) · SIAM Review · doi:10.1137/s0036144503423264 · Cited by 744
Gossip Algorithms for Distributed Signal Processing
Dimakis, A. G., Kar, S., Moura, J. M. F., Rabbat, M. G., & Scaglione, A. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2052531 · Cited by 741
Opinion Dynamics and Learning in Social Networks
Acemoglu, D., & Ozdaglar, A. (2011) · Dynamic Games and Applications · doi:10.1007/s13235-010-0004-1 · Cited by 705
Gossip Algorithms for Distributed Signal Processing
Dimakis, A. G., Kar, S., Moura, J. M. F., Rabbat, M., & Scaglione, A. (2018) · doi:10.1184/r1/6468842.v1 · Cited by 692
Kar, S., & Moura, J. M. F. (2009) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.2007111 · Cited by 691
Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks
Chen, J., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2198470 · Cited by 680
Self-citationKalman-Consensus Filter : Optimality, stability, and performance
Olfati-Saber, R. (2009) · Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference · doi:10.1109/cdc.2009.5399678 · Cited by 677
The Structure and Function of Fish Schools
Partridge, B. L. (1982) · Scientific American · doi:10.1038/scientificamerican0682-114 · Cited by 631
Diffusion recursive least-squares for distributed estimation over adaptive networks
Cattivelli, F. S., Lopes, C. G., & Sayed, A. H. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.913164 · Cited by 623
Self-citationStochastic Approximation and Recursive Estimation
Nevel′son, M., & Has′minskiĭ, R. (1976) · Translations of Mathematical Monographs · doi:10.1090/mmono/047 · Cited by 525
Jadbabaie, A., Molavi, P., Sandroni, A., & Tahbaz-Salehi, A. (2012) · Games and Economic Behavior · doi:10.1016/j.geb.2012.06.001 · Cited by 485
Distributing the Kalman Filter for Large-Scale Systems
Khan, U. A., & Moura, J. M. F. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.927480 · Cited by 481
Tu, S.-Y., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2217338 · Cited by 467
Self-citationDiffusion Adaptation Over Networks
Sayed, A. H. (2014) · Academic Press Library in Signal Processing · doi:10.1016/b978-0-12-411597-2.00009-6 · Cited by 465
Self-citationConvergence analysis of LMS filters with uncorrelated Gaussian data
Feuer, A., & Weinstein, E. (1985) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1985.1164493 · Cited by 375
Distributed learning in wireless sensor networks
Predd, J. B., Kulkarni, S. B., & Poor, H. V. (2006) · IEEE Signal Processing Magazine · doi:10.1109/msp.2006.1657817 · Cited by 369
Quorum responses and consensus decision making
Sumpter, D. J. T., & Pratt, S. C. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2008.0204 · Cited by 360
Distributed Asynchronous Constrained Stochastic Optimization
Srivastava, K., & Nedic, A. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2118740 · Cited by 350
Gardner, W. A. (1984) · Signal Processing · doi:10.1016/0165-1684(84)90013-6 · Cited by 343
The Perron-Frobenius theorem: some of its applications
Pillai, S. U., Suel, T., & Cha, S. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1406483 · Cited by 338
Subgradient methods and consensus algorithms for solving convex optimization problems
Johansson, B., Keviczky, T., Johansson, M., & Johansson, K. H. (2008) · 2008 47th IEEE Conference on Decision and Control · doi:10.1109/cdc.2008.4739339 · Cited by 316
Distributed Learning in Wireless Sensor Networks
Predd, J. B., Kulkarni, S. R., & Poor, H. V. (2007) · Wireless Sensor Networks · doi:10.1002/9780470061794.ch8 · Cited by 264
Kar, S., & Moura, J. M. F. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2127446 · Cited by 231
Aerodynamic aspects of formation flight in birds
Hummel, D. (1983) · Journal of Theoretical Biology · doi:10.1016/0022-5193(83)90110-8 · Cited by 186
Cooperative distributed multi-agent optimization
Nedić, A., & Ozdaglar, A. (2009) · Convex Optimization in Signal Processing and Communications · doi:10.1017/cbo9780511804458.011 · Cited by 185
Diffusion Adaptation Over Networks Under Imperfect Information Exchange and Non-Stationary Data
Zhao, X., Tu, S.-Y., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2192928 · Cited by 183
Self-citationTu, S.-Y., & Sayed, A. H. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2125943 · Cited by 182
Self-citationDistributed Pareto Optimization via Diffusion Strategies
Chen, J., & Sayed, A. H. (2013) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2013.2246763 · Cited by 177
Self-citationAdaptive Robust Distributed Learning in Diffusion Sensor Networks
Chouvardas, S., Slavakis, K., & Theodoridis, S. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2161474 · Cited by 165
Adaptive Processing: The Least Mean Squares Approach with Applications in Transmission
Macchi, O. (1995) · Medical Entomology and Zoology · Cited by 164
Information transfer in moving animal groups
Sumpter, D., Buhl, C., Biro, D., & Couzin, I. (2008) · Theory in Biosciences · doi:10.1007/s12064-008-0040-1 · Cited by 163
Modeling Bird Flight Formations Using Diffusion Adaptation
Cattivelli, F. S., & Sayed, A. H. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2107907 · Cited by 156
Self-citationA space-time diffusion scheme for peer-to-peer least-squares estimation
Xiao, L., Boyd, S., & Lall, S. (2006) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/1127777.1127806 · Cited by 155
Decentralized Parameter Estimation by Consensus Based Stochastic Approximation
Stankovic, S. S., Stanković, M. S., & Stipanović, D. M. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2076530 · Cited by 153
Performance Limits for Distributed Estimation Over LMS Adaptive Networks
Zhao, X., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2204985 · Cited by 146
Self-citationBio-Inspired Sensor Network Design
Barbarossa, S., & Scutari, G. (2007) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.361599 · Cited by 134
Adaptive Processing over Distributed Networks
Sayed, A. H., & Lopes, C. G. (2007) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1093/ietfec/e90-a.8.1504 · Cited by 131
Self-citationAnalysis of Spatial and Incremental LMS Processing for Distributed Estimation
Cattivelli, F. S., & Sayed, A. H. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2100386 · Cited by 130
Self-citationHow does an informed minority of scouts guide a honeybee swarm as it flies to its new home?
Beekman, M., Fathke, R. L., & Seeley, T. D. (2005) · Animal Behaviour · doi:10.1016/j.anbehav.2005.04.009 · Cited by 113
Steady-State Analysis of Diffusion LMS Adaptive Networks With Noisy Links
Khalili, A., Tinati, M. A., Rastegarnia, A., & Chambers, J. A. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2173338 · Cited by 108
Diffusion Least-Mean Squares Over Adaptive Networks
Lopes, C. G., & Sayed, A. H. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366830 · Cited by 108
Self-citationDistributed processing over adaptive networks
Sayed, A. H., & Lopes, C. G. (2007) · International Symposium on Signal Processing and Its Applications (ISSPA) · doi:10.1109/isspa.2007.4555636 · Cited by 90
Self-citationHoneybee swarms: how do scouts guide a swarm of uninformed bees?
Janson, S., Middendorf, M., & Beekman, M. (2005) · Animal Behaviour · doi:10.1016/j.anbehav.2004.10.018 · Cited by 88
Performance Analysis of the Consensus-Based Distributed LMS Algorithm
Mateos, G., Schizas, I. D., & Giannakis, G. B. (2009) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2009/981030 · Cited by 76
Clustering via diffusion adaptation over networks
Zhao, X., & Sayed, A. H. (2012) · International Workshop on Cognitive Information Processing (CIP) · doi:10.1109/cip.2012.6232902 · Cited by 75
Self-citationLearning over social networks via diffusion adaptation
Zhao, X., & Sayed, A. H. (2012) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2012.6489103 · Cited by 72
Self-citationA diffusion rls scheme for distributed estimation over adaptive networks
Cattivelli, F. S., Lopes, C. G., & Sayed, A. H. (2007) · IEEE Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2007.4401393 · Cited by 56
Self-citationOptimal Distributed Online Prediction
Dekel, O., Gilad-Bachrach, R., Shamir, O., & Xiao, L. (2011) · Cited by 53
Analysis of error-gradient adaptive linear estimators for a class of stationary dependent processes
Jones, S., Cavin, R. K., & Reed, W. R. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056477 · Cited by 43
On the Influence of Informed Agents on Learning and Adaptation Over Networks
Tu, S.-Y., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2230167 · Cited by 42
Self-citationDiffusion LMS algorithms with information exchange
Cattivelli, F. S., & Sayed, A. H. (2008) · 2018 52nd Asilomar Conference on Signals, Systems, and Computers · doi:10.1109/acssc.2008.5074402 · Cited by 38
Self-citationDiffusion Mechanisms For Fixed-Point Distributed Kalman Smoothing
Cattivelli, F. S., & Sayed, A. H. (2008) · doi:10.5281/zenodo.41205 · Cited by 33
Self-citationOn the limiting behavior of distributed optimization strategies
Chen, J., & Sayed, A. H. (2012) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2012.6483402 · Cited by 33
Self-citationSpatio-Temporal Diffusion Strategies for Estimation and Detection Over Networks
Lee, J.-W., Kim, S., Song, W.-J., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2197205 · Cited by 31
Self-citationBianchi, P., Fort, G., Hachem, W., & Jakubowicz, J. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947170 · Cited by 19
Distributed pareto-optimal solutions via diffusion adaptation
Chen, J., & Sayed, A. H. (2012) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2012.6319784 · Cited by 18
Self-citationDiffusion networks outperform consensus networks
Tu, S.-Y., & Sayed, A. H. (2012) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2012.6319691 · Cited by 12
Self-citationOn the generalization ability of distributed online learners
Towfic, Z. J., Chen, J., & Sayed, A. H. (2012) · IEEE International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2012.6349778 · Cited by 10
Self-citationOn the effects of topology and node distribution on learning over complex adaptive networks
Tu, S.-Y., & Sayed, A. H. (2011) · 2018 52nd Asilomar Conference on Signals, Systems, and Computers · doi:10.1109/acssc.2011.6190198 · Cited by 5
Self-citationEffective information flow over mobile adaptive networks
Tu, S.-Y., & Sayed, A. H. (2012) · International Workshop on Cognitive Information Processing (CIP) · doi:10.1109/cip.2012.6232903 · Cited by 4
Self-citationDistributed throughput optimization over P2P mesh networks using diffusion adaptation
Towfic, Z. J., Chen, J., & Sayed, A. H. (2012) · International Conference on Communications · doi:10.1109/icc.2012.6364457 · Cited by 3
Self-citation
Research works printed but not linked: 4
- theodoridis, 2008, Pattern Recognition
- horn, 2003, Matrix Analysis
- surowiecki, 2004, The Wisdom of Crowds
- chamley, 2004, Rational Herds
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Typical decentralized aggregation approaches include the consensus-based methods [69] and diffusion strategies [70].”
BackgroundMethodSayed, A. H. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2306253 · Cited by 587
Self-citationFederated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks
Savazzi, S., Nicoli, M., & Rampa, V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2964162 · Cited by 431
Adaptation, Learning, and Optimization over Networks
Sayed, A. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988515 · Cited by 415
Self-citationFrom distributed machine learning to federated learning: a survey
Liu, J., Huang, J., Zhou, Y., Li, X., Ji, S., Xiong, H., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01664-x · Cited by 356
Metadata
- Authors
- Sayed, A. H., Tu, S.-Y., Chen, J., Zhao, X., & Towfic, Z. J.
- Affiliations
- University of California, Los Angeles (US)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2013-05
- Volume
- 30
- Issue
- 3
- Pages
- 155-171
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W2042664989
- Cited by
- 494
- References
- 85
- Citations by year
- 2026: 12 · 2025: 16 · 2024: 33 · 2023: 32 · 2022: 31 · 2021: 36 · 2020: 47 · 2019: 34 · 2018: 47 · 2017: 45 · 2016: 52 · 2015: 42 · 2014: 43 · 2013: 21
- FWCI
- 52.20
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Signal Processing › Advanced Adaptive Filtering Techniques
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- diffusion strategies, distributed learning, network adaptation, emergent behavior, biological networks, swarming, fish schooling, bee swarms, multi-agent systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.