Distributed asynchronous deterministic and stochastic gradient optimization algorithms
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
We present a model for asynchronous distributed computation and then proceed to analyze the convergence of natural asynchronous distributed versions of a large class of deterministic and stochastic gradient-like algorithms. We show that such algorithms retain the desirable convergence properties of their centralized counterparts, provided that the time between consecutive interprocessor communications and the communication delays are not too large.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Theory and Practice of Recursive Identification
Lennart Ljung, A., & Söderström, T. (1983) · Lund University Publications (Lund University) · Cited by 3,989
Non-negative Matrices and Markov Chains
Seneta, E. (1981) · Springer Series in Statistics · doi:10.1007/0-387-32792-4 · Cited by 2,807
Analysis of recursive stochastic algorithms
Ljung, L. (1977) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1977.1101561 · Cited by 1,467
Stochastic Approximation Methods for Constrained and Unconstrained Systems
Kushner, H. J., & Clark, D. S. (1978) · Applied Mathematical Sciences · doi:10.1007/978-1-4684-9352-8 · Cited by 1,459
PROBLEMS IN DECENTRALIZED DECISION MAKING AND COMPUTATION
Τσιτσικλής, Ι. (2014) · doi:10.12681/eadd/3778 · Cited by 1,278
Some complexity questions related to distributive computing(Preliminary Report)
Yao, A. C.-C. (1979) · Proceedings of the eleventh annual ACM symposium on Theory of computing - STOC '79 · doi:10.1145/800135.804414 · Cited by 1,042
Rosenkrantz, W. A., & Meyer, P. A. (1967) · Mathematics of Computation · doi:10.2307/2004188 · Cited by 942
A Minimum Delay Routing Algorithm Using Distributed Computation
Gallager, R. (1977) · IEEE Transactions on Communications · doi:10.1109/tcom.1977.1093711 · Cited by 651
Asynchronous Iterative Methods for Multiprocessors
Baudet, G. M. (1978) · Journal of the ACM · doi:10.1145/322063.322067 · Cited by 519
Chazan, D., & Miranker, W. (1969) · Linear Algebra and its Applications · doi:10.1016/0024-3795(69)90028-7 · Cited by 460
Kendall, D. G. (1967) · Journal of the London Mathematical Society · doi:10.1112/jlms/s1-42.1.760 · Cited by 322
Solo, V. (1979) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1979.1102183 · Cited by 250
Distributed asynchronous computation of fixed points
Bertsekas, D. P. (1983) · Mathematical Programming · doi:10.1007/bf02591967 · Cited by 230
Self-citationDistributed dynamic programming
Bertsekas, D. (1982) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1982.1102980 · Cited by 216
Self-citationNollau, V. (1980) · ZAMM ‐ Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik · doi:10.1002/zamm.19800600133 · Cited by 122
Avriel, M. (1976) · Cited by 115
Koo, D. (1977) · Heidelberg science library · doi:10.1007/978-1-4612-6358-6_7 · Cited by 108
Synchronized and asynchronous parallel algorithms for multiprocessors
Kung, H. T. (2011) · Figshare · doi:10.1184/r1/6610166 · Cited by 87
On the complexity of designing distributed protocols
Papadimitriou, C. H., & Tsitsiklis, J. N. (1982) · Information and Control · doi:10.1016/s0019-9958(82)91034-8 · Cited by 45
Self-citationConvergence theories of distributed iterative processes: A survey
Bertsekas, D. P., Tsitsiklis, J. N., & Athans, M. (1986) · Lecture notes in control and information sciences · doi:10.1007/bfb0006867 · Cited by 26
Self-citationOptimal routing and flow control methods for communication networks
Bertsekas, D. P. (2006) · Springer eBooks · doi:10.1007/bfb0044418 · Cited by 25
Self-citationAnalysis and Optimization of Systems
(1982) · Lecture notes in control and information sciences · doi:10.1007/bfb0044376 · Cited by 12
Wallace, M. G. (2020) · Springer eBooks · doi:10.1007/978-3-030-41732-1_5 · Cited by 9
Distributed asynchronous optimal routing in data networks
Tsitsiklis, J. N., & Bertsekas, D. P. (1984) · doi:10.1109/cdc.1984.272190 · Cited by 6
Self-citation
Research works printed but not linked: 4
- poljak, 1973, Pseudogradient adaptation and training algorithms, Automat Remote Contr, 45
- bertsekas, 1986, Data Networks
- 1986, IEEE Trans Automat Contr, ac 31, 325
- arrow, 1960, Essays in Economics and Econometrics, 34
Cited by
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Consensus and Cooperation in Networked Multi-Agent Systems
Olfati-Saber, R., Fax, J. A., & Murray, R. M. (2007) · Proceedings of the IEEE · doi:10.1109/jproc.2006.887293 · Cited by 10,522
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Coverage Control for Mobile Sensing Networks
Cortes, J., Martinez, S., Karatas, T., & Bullo, F. (2004) · IEEE Transactions on Robotics and Automation · doi:10.1109/tra.2004.824698 · Cited by 2,710
Metadata
- Authors
- Tsitsiklis, J., Bertsekas, D., & Athans, M.
- Affiliations
- Massachusetts Institute of Technology (US)
- Venue
- IEEE Transactions on Automatic Control
- Published
- 1986-09
- Volume
- 31
- Issue
- 9
- Pages
- 803-812
- ISSN
- 0018-9286 (ISSN-L)
- OpenAlex
- W2154834860
- Cited by
- 1,967
- References
- 24
- Citations by year
- 2026: 28 · 2025: 63 · 2024: 60 · 2023: 85 · 2022: 71 · 2021: 116 · 2020: 114 · 2019: 107 · 2018: 125 · 2017: 106 · 2016: 113 · 2015: 126 · 2014: 133 · 2013: 123 · 2012: 127
- FWCI
- 7.50
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Automatic Control (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Mathematics › Numerical Analysis › Advanced Optimization Algorithms Research
- Keywords
- stochastic gradient optimization, convergence analysis, communication delay, interprocessor communication, distributed optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.