The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Speeding Up Distributed Machine Learning Using Codes
Open accessAbstract
Codes are widely used in many engineering applications to offerrobustnessagainstnoise. In large-scale systems, there are several types of noise that can affect the performance of distributed machine learning algorithms—straggler nodes, system failures, or communication bottlenecks—but there has been little interaction cutting across codes, machine learning, and distributed systems. In this paper, we provide theoretical insights on howcodedsolutions can achieve significant gains compared with uncoded ones. We focus on two of the most basic building blocks of distributed learning algorithms:matrix multiplicationanddata shuffling. For matrix multiplication, we use codes to alleviate the effect of stragglers and show that if the number of homogeneous workers is$n$, and the runtime of each subtask has an exponential tail, coded computation can speed up distributed matrix multiplication by a factor of$\log n$. For data shuffling, we use codes to reduce communication bottlenecks, exploiting the excess in storage. We show that when a constant fraction$\alpha $of the data matrix can be cached at each worker, and$n$is the number of workers,coded shufflingreduces the communication cost by a factor of$\left({\alpha + \frac {1}{n}}\right)\gamma (n)$compared with uncoded shuffling, where$\gamma (n)$is the ratio of the cost of unicasting$n$messages to$n$users to multicasting a common message (of the same size) to$n$users. For instance,$\gamma (n) \simeq n$if multicasting a message to$n$users is as cheap as unicasting a message to one user. We also provide experimental results, corroborating our theoretical gains of the coded algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Data poisoning attacks may occur in two ways such as clean-label attack (adversary can poison correct class of data sample) and dirty-label attack (adversary try to misclassify the target label of training dataset) [147], [148].”
From Challenges, Applications and Design Aspects of Federated Learning: A Survey (2021) · cites this paper as [148]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,917
Smith, D. K., & Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 12,385
(2010) · Cited by 4,411
Flagged first, then by citations
Research works printed but not linked: 3
- 10.1109/ISIT.2016.7541478 · DOI
- 2015, StarCluster
- 2015, Open MPI Open Source High Performance Computing
Cited by
Sorted by citations
Items 801–827 of 827
FedTeddi: Temporal Drift and Divergence Aware Scheduling for Timely Federated Edge Learning
Bai, Y., Sun, Y., Chen, T., Chen, W., Zhou, S., & Niu, Z. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11322874
Straggler Dynamic Management for Distributed DNN Training
Li, T., Chai, B., Tan, X., Yuan, S., Ju, K., & Zhu, S. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323136
Fundamental Limits of Multi-User Distributed Computing of Linearly Separable Functions
Namboodiri, K. K. K., Peter, E., Malak, D., & Elia, P. (2026) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit62367.2026.11654055
Paolini, E., Liva, G., Matuz, B., & Chiani, M. (2026) · doi:10.1017/9781316443354.013
Fundamental Limits of Distributed Linearly Separable Computation Under Cyclic Assignment
Huang, W., Wan, K., Sun, H., Ji, M., Qiu, R. C., & Caire, G. (2026) · IEEE Transactions on Communications · doi:10.1109/tcomm.2026.3669452
SpotCC: Facilitating Coded Computation for Prediction Serving Systems on Spot Instances
Wang, L., Hu, Y., Duan, Z., Li, M., Yao, C., Liu, F., Li, X., Qin, L., & Feng, D. (2026) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca68181.2026.11408557
CEC-DuelNet: Relational Deep Reinforcement Learning for Coded Edge Computing Offloading
Narayan, U., Priyadarshni, Praveen Kumar, I., Tripathi, S., & Misra, R. N. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401676
Minimum Cost Encoding for Coded Distributed Computing Systems
Karami, M., Ardakani, M., Ebrahimzad, H., & Zhang, Z. (2026) · IEEE Transactions on Communications · doi:10.1109/tcomm.2026.3677353
DMS: Throughput-Aware Distributed Multimodal Model Surgery on Embedded Devices
Cai, Q., Li, K., Zhang, K., Xie, X. N., Tong, X., & Li, K. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3682748
General Coded Computing: Where Coding Theory Meets Learning Theory
Moradi, P., & Maddah-Ali, M. A. (2025) · IEEE BITS the Information Theory Magazine · doi:10.1109/mbits.2026.3693276
Approximate Distributed Coded Computing: Polynomial Codes and Randomized Sketching
Charalambides, N., & Mazumdar, A. (2025) · IEEE BITS the Information Theory Magazine · doi:10.1109/mbits.2026.3694647
Coded Data Rebalancing for Distributed Data Storage Systems With Cyclic Storage
Vaishya, A., Chandramouli, A., Kale, S., & Krishnan, P. (2026) · IEEE Transactions on Information Theory · doi:10.1109/tit.2026.3700033
Heterogeneous Multi-Computing-Units Based LT Coded Computation in Wireless Networks
Fang, B., & Qiao, X. (2026) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc65185.2026.11555635
Cost-Constrained Node Selection for Timely Coded Computation Over Heterogeneous Systems
Tang, B., Lin, Y., Zhou, S., Xie, Z., & Ye, B. (2026) · IEEE Transactions on Computers · doi:10.1109/tc.2026.3705724
FedACT: Concurrent Federated Intelligence across Heterogeneous Data Sources
Islam, M. S., Chapman, I. G., Ashafuddula, N. I. M., Yuan, X., Chen, L., Tzeng, N.-F., & Nahrstedt, K. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00065
Multivariate Polynomial Codes for Efficient Matrix Chain Multiplication in Distributed Systems
Gómez-Vilardebó, J. (2026) · International Conference on Communications · doi:10.1109/icc59461.2026.11587322
FVCC: Enabling Fast and Verifiable Coded Computation for Robust Distributed Learning
Wang, L., Xu, L., Liu, Y., Yang, X., Li, M., & Wang, J. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3723143
Information-Theoretic Groupwise-Key Secure Aggregation under User Selection and Collusion
Qin, J., Li, Z., & Zhao, Y. (2026) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit62367.2026.11654114
FedRIDA: Coordinated Participation and Update Control for Time-Efficient Federated Learning
Yan, G., & Jia, Y. (2026) · International Workshop on Quality of Service · doi:10.1109/iwqos70441.2026.11661126
Coded Redundancy for Reducing Task Average Latency in Distributed Stream Processing Systems
Lin, Y., Tang, B., Wang, S., Chen, L., Jiang, X., & Buyya, R. (2026) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud72782.2026.00026
Pairwise Aggregation with Cyclic Storage
Huang, W., Zhang, M., Wan, K., Cheng, M., & Qiu, R. C. (2026) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccc70295.2026.11680212
Ardakani, M. H., Heidarpour, A. R., Ardakani, M., Tellambura, C., Leon‐Garcia, A., & Uysal, M. (2026) · IEEE Communications Letters · doi:10.1109/lcomm.2026.3741039
Robust and communication-efficient collaborative learning
Reisizadeh, A., Taheri, H., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1907.10595 · Cited by 73
Self-citation
Metadata
- Authors
- Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- University of California, Berkeley (US)
- University of California, Santa Barbara (US)
- University of Wisconsin–Madison (US)
- Venue
- IEEE Transactions on Information Theory
- Published
- 2018-03
- Volume
- 64
- Issue
- 3
- Pages
- 1514-1529
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2268702383
- Cited by
- 829
- References
- 122
- Citations by year
- 2026: 24 · 2025: 56 · 2024: 79 · 2023: 101 · 2022: 111 · 2021: 148 · 2020: 130 · 2019: 105 · 2018: 48
- FWCI
- 54.46
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Information Theory (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute for Information & communications Technology Promotion(IITP) grant
- Korea government(MSIT) (2017-0-00694)
- Brain Korea 21 Plus Project, and NSF CIF grant (1703678)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- distributed learning, coded computing, straggler nodes, distributed matrix multiplication, data shuffling, communication bottleneck, multicast, caching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
