Slow and Stale Gradients Can Win the Race
Open accessThe DOI matches its Crossref record; 90 of the 90 references it cites were found in a registry or an index.
Abstract
Distributed Stochastic Gradient Descent (SGD) when run in a synchronous manner, suffers from delays inruntimeas it waits for the slowest workers (stragglers). Asynchronous methods can alleviate stragglers, but cause gradient staleness that can adversely affect the convergenceerror. In this work, we present a novel theoretical characterization of the speedup offered by asynchronous methods by analyzing the trade-off between theerrorin the trained model and the actual trainingruntime(wallclock time). The main novelty in our work is that our runtime analysis considers random straggling delays, which helps us design and compare distributed SGD algorithms that strike a balance between straggling and staleness. We also provide a new error convergence analysis of asynchronous SGD variants without bounded or exponential delay assumptions. Finally, based on our theoretical characterization of the error-runtime trade-off, we propose a method of gradually varying synchronicity in distributed SGD and demonstrate its performance on the CIFAR10 dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Cross-device federated learning and federated data analysis are now being applied in consumer digital products.”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [112]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Google makes extensive use of federated learning in the Gboard mobile keyboard [376, 222, 491, 112, 383], as well as in features on Pixel phones [14] and in Android Messages [439].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [112]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“When local datasets are small and the data is IID, FL clearly has an edge, and indeed, real-world applications of federated learning [491, 222, 112] benefit from training a single model across devices.”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [112]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
(2010) · Cited by 8,261
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
An overview of gradient descent optimization algorithms
Ruder, S. (2016) · arXiv · doi:10.48550/arxiv.1609.04747 · Cited by 4,962
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
Distributed asynchronous deterministic and stochastic gradient optimization algorithms
Tsitsiklis, J., Bertsekas, D., & Athans, M. (1986) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1986.1104412 · Cited by 1,967
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Johnson, R., & Zhang, T. (2013) · Cited by 1,889
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852
A First Course in Probability.
Sherman, M. J., & Ross, S. (1984) · Journal of the American Statistical Association · doi:10.2307/2288753 · Cited by 1,344
A Course in Microeconomic Theory
Kreps, D. M. (1990) · Princeton University Press eBooks · doi:10.1515/9780691215747 · Cited by 1,324
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211
Revisiting Distributed Synchronous SGDPreprint
Pan, X.-H., Chen, J.-M., Monga, R., Bengio, S., & Józefowicz, R. (2016) · arXiv.org · doi:10.48550/arxiv.1604.00981 · Cited by 865
Speeding Up Distributed Machine Learning Using Codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2736066 · Cited by 829
Efficient mini-batch training for stochastic optimization
Li, M., Zhang, T., Chen, Y., & Smola, A. J. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623612 · Cited by 742
Revisiting Distributed Synchronous SGD
Pan, X., Chen, J., Monga, R., Bengio, S., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 548
Deep learning with Elastic Averaging SGDPreprint
Zhang, S., Choromanska, A., & LeCun, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6651 · Cited by 546
Optimal Distributed Online Prediction using Mini-Batches
Dekel, O., Gilad-Bachrach, R., Shamir, O., & Xiao, L. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.1367 · Cited by 512
More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server
Ho, Q., Cipar, J., Cui, H., Kim, J. K., Lee, S., Gibbons, P. B., Gibson, G. A., Ganger, G. R., & Xing, E. P. (2013) · PubMed · doi:10.1184/r1/6475898 · Cited by 506
A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets
Roux, N. L., Schmidt, M., & Bach, F. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1202.6258 · Cited by 481
A Fundamental Tradeoff Between Computation and Communication in Distributed Computing
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2756959 · Cited by 451
The method of steepest descent for non-linear minimization problems
Curry, H. B. (1944) · Quarterly of Applied Mathematics · doi:10.1090/qam/10667 · Cited by 437
Gradient Coding: Avoiding Stragglers in Distributed Learning
Tandon, R., Lei, Q., Dimakis, A. G., & Karampatziakis, N. (2017) · International Conference on Machine Learning · Cited by 431
Stochastic Processes: Theory for Applications
Gallager, R. G. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9781139626514 · Cited by 417
Stochastic Processes: Theory for Applications
Gallager, R. G. (2014) · Medical Entomology and Zoology · Cited by 370
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive GradientPreprint
Nguyen, L. M., Liu, J., Scheinberg, K., & Takáč, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00102 · Cited by 325
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
Lian, X., Huang, Y., Li, Y., & Liu, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.08272 · Cited by 324
“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products
Dutta, S., Cadambe, V., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2927558 · Cited by 313
Self-citationZhou, F., & Cong, G. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/447 · Cited by 255
ARock: An Algorithmic Framework for Asynchronous Parallel Coordinate Updates
Peng, Z., Xu, Y., Yan, M., & Yin, W. (2016) · SIAM Journal on Scientific Computing · doi:10.1137/15m1024950 · Cited by 247
Distributed delayed stochastic optimization
Agarwal, A., & Duchi, J. C. (2012) · 2012 IEEE 51st IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426626 · Cited by 243
Communication-Computation Efficient Gradient CodingPreprint
Ye, M., & Abbe, E. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03475 · Cited by 203
On the Delay-Storage Trade-Off in Content Download from Coded Distributed Storage Systems
Joshi, G., Liu, Y., & Soljanin, E. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.140518 · Cited by 186
Self-citationAdaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGDPreprint
Wang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08313 · Cited by 175
Self-citationStaleness-aware Async-SGD for Distributed Deep Learning
Zhang, W., Gupta, S., Lian, X., & Liu, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05950 · Cited by 171
Improving Distributed Gradient Descent Using Reed-Solomon Codes
Halbawi, W., Azizan, N., Salehi, F., & Hassibi, B. (2018) · 2018 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437467 · Cited by 157
Perturbed Iterate Analysis for Asynchronous Stochastic Optimization
Mania, H., Pan, X., Papailiopoulos, D., Recht, B., Ramchandran, K., & Jordan, M. I. (2017) · SIAM Journal on Optimization · doi:10.1137/16m1057000 · Cited by 154
Addressing the straggler problem for iterative convergent parallel ML
Harlap, A., Cui, H., Dai, W., Wei, J., Ganger, G. R., Gibbons, P. B., Gibson, G. A., & Xing, E. P. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987554 · Cited by 154
Efficient Redundancy Techniques for Latency Reduction in Cloud Systems
Joshi, G., Soljanin, E., & Wornell, G. W. (2017) · ACM Transactions on Modeling and Performance Evaluation of Computing Systems · doi:10.1145/3055281 · Cited by 152
Self-citationMATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling
Wang, J., Sahu, A. K., Yang, Z., Joshi, G., & Kar, S. (2019) · 2019 Sixth Indian Control Conference (ICC) · doi:10.1109/icc47138.2019.9123209 · Cited by 149
Self-citationWang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07576 · Cited by 146
Self-citationModel Accuracy and Runtime Tradeoff in Distributed Deep Learning: A Systematic Study
Gupta, S., Zhang, W., & Wang, F. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0028 · Cited by 139
Coded convolution for parallel and distributed computing within a deadline
Dutta, S., Cadambe, V. R., & Grover, P. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006960 · Cited by 134
Self-citationUsing Straggler Replication to Reduce Latency in Large-scale Parallel Computing
Wang, D., Joshi, G., & Wornell, G. W. (2015) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2847220.2847223 · Cited by 126
Self-citationSlow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGDPreprint
Dutta, S., Joshi, G., Ghosh, S., Dube, P., & Nagpurkar, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01113 · Cited by 125
Self-citationExploiting bounded staleness to speed up Big Data analytics
Cui, H., Cipar, J., Ho, Q., Kim, J. K., Lee, S., Kumar, A., Wei, J., Dai, W., Ganger, G. R., Gibbons, P. B., et al. (2018) · doi:10.1184/r1/6475613 · Cited by 119
Gradient Diversity: a Key Ingredient for Scalable Distributed LearningPreprint
Yin, D., Pananjady, A., Lam, M. W. Y., Papailiopoulos, D., Ramchandran, K., & Bartlett, P. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05699 · Cited by 119
Straggler Mitigation in Distributed Optimization Through Data EncodingPreprint
Karakus, C., Sun, Y., Diggavi, S. N., & Yin, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04969 · Cited by 118
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication
Mallick, A., Chaudhari, M., Sheth, U., Palanikumar, G., & Joshi, G. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366706 · Cited by 115
Self-citationAsynchrony begets momentum, with an application to deep learning
Mitliagkas, I., Zhang, C., Hadjis, S., & Re, C. (2016) · 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton) · doi:10.1109/allerton.2016.7852343 · Cited by 113
A Unified Coded Deep Neural Network Training Strategy based on Generalized PolyDot codes
Dutta, S., Bai, Z., Jeong, H., Low, T. M., & Grover, P. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437852 · Cited by 103
Self-citationDistributed Delayed Stochastic Optimization
Agarwal, A., & Duchi, J. C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1104.5525 · Cited by 95
Solving the Straggler Problem with Bounded Staleness
Cipar, J., Ho, Q., Kim, J. K., Lee, S., Ganger, G. R., Gibson, G. A., Keeton, K., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6476339 · Cited by 87
Speeding Up Distributed Gradient Descent by Utilizing Non-persistent Stragglers
Özfatura, E., Gündüz, D., & Ulukuş, Ş. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849684 · Cited by 82
On the optimal recovery threshold of coded matrix multiplication
Fahim, M., Jeong, H., Haddadpour, F., Dutta, S., Cadambe, V. R., & Grover, P. (2017) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2017.8262882 · Cited by 81
Self-citationAsynchronous stochastic convex optimization: the noise is in the noise and SGD don't care
Chaturapruek, S., Duchi, J. C., & Ré, C. (2015) · neural information processing systems · Cited by 80
Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization
Haddadpour, F., Kamani, M. M., Mahdavi, M., & Cadambe, V. R. (2019) · Cited by 79
Parallel SGD: When does averaging help?
Zhang, J., De, C., Mitliagkas, I., & Ré, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.07365 · Cited by 73
Robust and communication-efficient collaborative learning
Reisizadeh, A., Taheri, H., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1907.10595 · Cited by 73
Asynchronous Coordinate Descent under More Realistic AssumptionsPreprint
Sun, T., Hannah, R., & Yin, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08494 · Cited by 68
Coded distributed computing for inverse problems
Yang, Y. L., Grover, P., & Kar, S. (2017) · neural information processing systems · Cited by 66
Robust Gradient Descent via Moment Encoding and LDPC Codes
Maity, R. K., Rawat, A. S., & Mazumdar, A. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849514 · Cited by 65
On Unbounded Delays in Asynchronous Parallel Fixed-Point Algorithms
Hannah, R., & Yin, W. (2017) · Journal of Scientific Computing · doi:10.1007/s10915-017-0628-z · Cited by 60
Encoded distributed optimization
Karakus, C., Sun, Y., & Diggavi, S. N. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8007058 · Cited by 55
Computation Scheduling for Distributed Machine Learning with Straggling Workers
Amiri, M. M., & Gunduz, D. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682911 · Cited by 49
ASAGA: Asynchronous Parallel SAGAPreprint
Leblond, R., Pedregosa, F., & Lacoste-Julien, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04809 · Cited by 47
Approximate Gradient Coding via Sparse Random Graphs
Charles, Z., Papailiopoulos, D., & Ellenberg, J. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06771 · Cited by 45
Fundamental Limits of Approximate Gradient Coding
Wang, S., Liu, J., & Shroff, N. B. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366700 · Cited by 43
Joshi, G., Soljanin, E., & Wornell, G. W. (2015) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2825236.2825258 · Cited by 41
Self-citationStraggler Mitigation by Delayed Relaunch of Tasks
Aktaş, M. F., Peng, P., & Soljanin, E. (2018) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3199524.3199564 · Cited by 39
MATCHA: Speeding Up Decentralized SGD via Matching Decomposition SamplingPreprint
Wang, J., Sahu, A. K., Yang, Z., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09435 · Cited by 38
Self-citationLi, S., Supittayapornpong, S., Maddah-Ali, M. A., & Avestimehr, S. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2017.33 · Cited by 38
Li, S., Supittayapornpong, S., Maddah-Ali, M. A., & Avestimehr, S. (2017) · Cited by 31
Sheth, U., Dutta, S., Chaudhari, M., Jeong, H., Yang, Y. L., Kohonen, J., Roos, T., & Grover, P. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622429 · Cited by 31
Self-citationYang, X., Huang, K., & Zhang, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70087-8_73 · Cited by 22
Hannah, R., & Yin, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05136 · Cited by 19
Fault-tolerant distributed logistic regression using unreliable components
Yang, Y. L., Grover, P., & Kar, S. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852335 · Cited by 17
Sequential addition of coded sub-tasks for straggler mitigation
Badita, A., Parag, P., & Aggarwal, V. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155358 · Cited by 10
Gradient Diversity Empowers Distributed Learning.Preprint
Yin, D., Pananjady, A., Lam, M., Papailiopoulos, D., Ramchandran, K., & Bartlett, P. L. (2017) · arXiv (Cornell University) · Cited by 5
Distributed SGD Generalizes Well Under Asynchrony
Regatti, J. R., Tendolkar, G., Zhou, Y., Gupta, A., & Liang, Y. (2019) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2019.8919791 · Cited by 4
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector MultiplicationPreprint
Mallick, A., Chaudhari, M., Sheth, U., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.10331 · Cited by 2
Federated Learning: Challenges, Methods, and Future Directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2019) · arXiv · doi:10.48550/arxiv.1908.07873
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2016) · arXiv · doi:10.48550/arxiv.1606.04838
Improving Distributed Gradient Descent Using Reed-Solomon Codes
Halbawi, W., Azizan-Ruhi, N., Salehi, F., & Hassibi, B. (2017) · arXiv · doi:10.48550/arxiv.1706.05436
Zhou, F., & Cong, G. (2017) · arXiv · doi:10.48550/arxiv.1708.01012
Research works printed but not linked: 2
- al-lawati, 2019, Anytime minibatch with stale gradients, Proc 53rd Annu Conf Inf Sci Syst (CISS), 1
- cadambe, 2017, Codes for distributed computing: A tutorial, IEEE Inf Theory Soc Newslett, 67, 3
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse GradientsPreprint
Mitra, A., Jaafar, R. H., Pappas, G. J., & Hassani, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07053 · Cited by 203
Multi-Stage Asynchronous Federated Learning With Adaptive Differential Privacy
Li, Y., Yang, S., Ren, X., Shi, L., & Zhao, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3332428 · Cited by 55
Stigmergic Independent Reinforcement Learning for Multiagent Collaboration
Xu, X., Li, R., Zhao, Z., & Zhang, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3056418 · Cited by 42
Robust Federated Learning for Unreliable and Resource-Limited Wireless Networks
Chen, Z., Yi, W., Liu, Y., & Nallanathan, A. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3366393 · Cited by 50
Metadata
- Authors
- Dutta, S., Wang, J., & Joshi, G.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- IEEE Journal on Selected Areas in Information Theory
- Published
- 2021-09
- Volume
- 2
- Issue
- 3
- Pages
- 1012-1024
- ISSN
- 2641-8770 (ISSN-L)
- OpenAlex
- W3013308737
- Cited by
- 40
- References
- 88
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 14 · 2023: 5 · 2022: 5 · 2021: 8
- FWCI
- 2.80
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Journal on Selected Areas in Information Theory (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- IBM Faculty Award
- National Science Foundation CRII Award (CCF-1717314)
- Qualcomm Innovation Fellowship
- AISTATS 2018
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- distributed stochastic gradient descent, asynchronous SGD, convergence analysis, random delay, distributed training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.