Communication Complexity of Distributed Convex Learning and Optimization

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,917

Introductory Lectures on Convex Optimization: A Basic Course

Нестеров, Ю. Е. (2014) · Medical Entomology and Zoology · Cited by 3,576

Smooth minimization of non-smooth functions

Nesterov, Y. E. (2004) · Mathematical Programming · doi:10.1007/s10107-004-0552-5 · Cited by 2,437

Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211

Parallelized Stochastic Gradient Descent

Zinkevich, M., Weimer, M., Li, L., & Smola, A. J. (2010) · Cited by 1,124

Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling

Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2010) · Cited by 884

Optimal Distributed Online Prediction using Mini-BatchesPreprint

DekelOfer, Gilad-BachrachRan, ShamirOhad, & Xiaolin (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.1367 · Cited by 564

Communication-Efficient Distributed Optimization using an Approximate Newton-type Method

Shamir, O., Srebro, N., & Zhang, T. (2013) · arXiv · doi:10.48550/arxiv.1312.7853 · Cited by 424

Numerical linear algebra in the streaming model

Clarkson, K. L., & Woodruff, D. P. (2009) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/1536414.1536445 · Cited by 304

Scaling up machine learning

Bekkerman, R., Bilenko, M., & Langford, J. C. (2011) · doi:10.1145/2107736.2107740 · Cited by 290

Communication-efficient distributed multi-task learning with matrix sparsity regularization

Zhou, Q., Chen, Y., & Pan, S. J. (2019) · Machine Learning · doi:10.1007/s10994-019-05847-6 · Cited by 4

Gradual Federated Learning Using Simulated Annealing

Nguyen, L. T., & Shim, B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414455 · Cited by 4

Optimal Data Splitting in Distributed Optimization for Machine Learning

Medyakov, D. O., Molodtsov, G. L., Beznosikov, A. N., & Гасников, А. В. (2023) · Doklady Mathematics · doi:10.1134/s1064562423701600 · Cited by 3

Improved Convergence Rates for Non-Convex Federated Learning with Compression.

Das, R., Hashemi, A., Sanghavi, S., & Dhillon, I. S. (2020) · Cited by 3

Efficient Convex Optimization Requires Superlinear Memory (Extended Abstract)

Marsden, A., Sharan, V., Sidford, A., & Valiant, G. J. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/722 · Cited by 3

A Provably Communication-Efficient Asynchronous Distributed Inference Method for Convex and Nonconvex Problems

Ren, J., & Haupt, J. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2996374 · Cited by 3

Model Aggregation via Good-Enough Model SpacesPreprint

Guha, N., & Smith, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07782 · Cited by 7

Towards Tight Communication Lower Bounds for Distributed OptimisationPreprint

Alistarh, D., & Korhonen, J. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08222 · Cited by 3

Efficient distributed optimization for large-scale high-dimensional sparse penalized Huber regression

Pan, Y., Xu, K., Wei, S., Wang, X., & Liu, Z. (2022) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2022.2098331 · Cited by 2

An Improved Video Recommendation System for IoT Devices Using Federated Learning

Kabir, M., Tasfia, R., & Mostafa, M. G. (2023) · International Conference on Device Intelligence, Computing and Communication Technologies (DICCT) · doi:10.1109/dicct56244.2023.10110183 · Cited by 2

CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problems

Xie, J., Zhang, C., Shen, Z., Liu, W., & Qian, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26246 · Cited by 11

Decentralized saddle point problems via non-Euclidean mirror prox

Rogozin, A., Beznosikov, A. N., Dvinskikh, D., Kovalev, D. A., Dvurechensky, P. E., & Gasnikov, A. V. (2024) · Optimization methods & software · doi:10.1080/10556788.2023.2280062 · Cited by 2

A Communication-Efficient Algorithm for Federated Multilevel Stochastic Compositional Optimization

Yang, S., & Li, F. (2024) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2024.3392351 · Cited by 2

Local methods with adaptivity via scaling

Chezhegov, S., Skorik, S., Khachaturov, N., Shalagin, D., Avetisyan, A. A., Takáč, M., Kholodov, Y. A., & Beznosikov, A. N. (2024) · Успехи математических наук · doi:10.4213/rm10209 · Cited by 2

Communication Lower Bounds for Distributed Convex Optimization: Partition Data on Features

Chen, Z., Luo, L., & Zhang, Z. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10912 · Cited by 2

Accelerated Sparsified SGD with Error FeedbackPreprint

Murata, T., & Suzuki, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12224 · Cited by 2

Collaborative Top Distribution Identifications with Limited InteractionPreprint

Karpov, N., Zhang, Q., & Zhou, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.09454 · Cited by 2

A Survey on Large-scale Machine LearningPreprint

Wang, M., Fu, W., He, X., Hao, S., & Wu, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03911 · Cited by 2

The Minimax Complexity of Distributed OptimizationPreprint

Woodworth, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.00534 · Cited by 2

Q-linear Convergence of Distributed Optimization with Barzilai-Borwein Step Sizes

Emiola, I., & Enyioha, C. (2022) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton49937.2022.9929424 · Cited by 1

On the Role of Data Homogeneity in Multi-Agent Non-convex Stochastic Optimization

Li, Q., & Wai, H.-T. (2022) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc51059.2022.9992403 · Cited by 1

Zero Order Algorithm for Decentralized Optimization Problems

Veprikov, A., Petrov, E. D., Evseev, G. V., & Beznosikov, A. N. (2024) · Doklady Mathematics · doi:10.1134/s1064562424602336 · Cited by 2

PROVABLY COMMUNICATION-EFFICIENT ASYNCHRONOUS DISTRIBUTED INFERENCE FOR CONVEX AND NONCONVEX PROBLEMS

Ren, J., & Haupt, J. (2018) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2018.8646525 · Cited by 1

Parallel and Distributed algorithms for ML problemsPreprint

Dvinskikh, D., Gasnikov, A. V., Rogozin, A., & Beznosikov, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09585 · Cited by 1

Delayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and ApplicationsPreprint

Li, X., & Zhang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01505 · Cited by 2

Decentralized Algorithms for Wasserstein Barycenters

Dvinskikh, D. (2021) · doi:10.18452/23600 · Cited by 1

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problem.

Yu, Y., Liu, S., & Pan, S. J. (2017) · Uncertainty in Artificial Intelligence

Trade-Offs in Large-Scale Distributed Tuplewise Estimation And Learning

Vogel, R., Bellet, A., Clémençon, S., Jelassi, O., & Papa, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46147-8_14

Machine Learning System Aware Optimization

-, P. S., & Sharma, P. (2024) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2024.v06i04.26655

Accelerated Stochastic ExtraGradient: Mixing Hessian and gradient similarity to reduce communication in distributed and federated learning

Bylinkin, D., Degtyarev, K., & Beznosikov, A. N. (2024) · Успехи математических наук · doi:10.4213/rm10206

Local methods with adaptivity via scaling

Chezhegov, S., Skorik, S., Khachaturov, N., Shalagin, D., Avetisyan, A. A., Takáč, M., Kholodov, Y. A., & Beznosikov, A. N. (2024) · Russian Mathematical Surveys · doi:10.4213/rm10209e

Accelerated Stochastic ExtraGradient: Mixing Hessian and gradient similarity to reduce communication in distributed and federated learning

Bylinkin, D., Degtyarev, K., & Beznosikov, A. N. (2024) · Russian Mathematical Surveys · doi:10.4213/rm10206e

A Unified Approach to Memory-Sample Tradeoffs for Detecting Planted Structures

Garg, S., Hastings, J., Pabbaraju, C., & Sharan, V. (2026) · ACM Symposium on Theory of Computing (STOC) · doi:10.1145/3798129.3800739

Towards Optimal Communication Complexity in Distributed Non-Convex Optimization

Patel, K. K., Wang, L., Woodworth, B., Bullins, B., & Srebro, N. (2022) · neural information processing systems · doi:10.52202/068431-0968

Communication Lower Bounds for Distributed Convex Optimization: Partition Data on Features

Chen, Z., Luo, L., & Zhang, Z. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10912 · Cited by 2

Improved Communication Lower Bounds for Distributed OptimisationPreprint

Korhonen, J. H., & Alistarh, D. (2021) · arXiv (Cornell University) · Cited by 2

Distributed Stochastic Variance Reduced Gradient Methods and A Lower Bound for Communication ComplexityPreprint

Lee, J. D., Lin, Q., Ma, T., & Yang, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.07595 · Cited by 38

The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent CommunicationPreprint

Woodworth, B., Bullins, B., Shamir, O., & Srebro, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01583 · Cited by 19