The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Speeding Up Distributed Machine Learning Using Codes
Open accessAbstract
Codes are widely used in many engineering applications to offerrobustnessagainstnoise. In large-scale systems, there are several types of noise that can affect the performance of distributed machine learning algorithms—straggler nodes, system failures, or communication bottlenecks—but there has been little interaction cutting across codes, machine learning, and distributed systems. In this paper, we provide theoretical insights on howcodedsolutions can achieve significant gains compared with uncoded ones. We focus on two of the most basic building blocks of distributed learning algorithms:matrix multiplicationanddata shuffling. For matrix multiplication, we use codes to alleviate the effect of stragglers and show that if the number of homogeneous workers is$n$, and the runtime of each subtask has an exponential tail, coded computation can speed up distributed matrix multiplication by a factor of$\log n$. For data shuffling, we use codes to reduce communication bottlenecks, exploiting the excess in storage. We show that when a constant fraction$\alpha $of the data matrix can be cached at each worker, and$n$is the number of workers,coded shufflingreduces the communication cost by a factor of$\left({\alpha + \frac {1}{n}}\right)\gamma (n)$compared with uncoded shuffling, where$\gamma (n)$is the ratio of the cost of unicasting$n$messages to$n$users to multicasting a common message (of the same size) to$n$users. For instance,$\gamma (n) \simeq n$if multicasting a message to$n$users is as cheap as unicasting a message to one user. We also provide experimental results, corroborating our theoretical gains of the coded algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Data poisoning attacks may occur in two ways such as clean-label attack (adversary can poison correct class of data sample) and dirty-label attack (adversary try to misclassify the target label of training dataset) [147], [148].”
From Challenges, Applications and Design Aspects of Federated Learning: A Survey (2021) · cites this paper as [148]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,917
Smith, D. K., & Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 12,385
(2010) · Cited by 4,411
Research works printed but not linked: 3
- 10.1109/ISIT.2016.7541478 · DOI
- 2015, StarCluster
- 2015, Open MPI Open Source High Performance Computing
Cited by
Sorted by citations
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587
Self-citationMachine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
A Fundamental Tradeoff Between Computation and Communication in Distributed Computing
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2756959 · Cited by 458
Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix MultiplicationPreprint
Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10464 · Cited by 439
Gradient Coding: Avoiding Stragglers in Distributed Learning
Tandon, R., Lei, Q., Dimakis, A. G., & Karampatziakis, N. (2017) · International Conference on Machine Learning · Cited by 431
Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning
Shi, W., Zhou, S., Niu, Z., Jiang, M., & Geng, L. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3025446 · Cited by 381
“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products
Dutta, S., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2927558 · Cited by 361
Lagrange Coded Computing: Optimal Design for Resiliency, Security and PrivacyPreprint
Yu, Q., Li, S., Raviv, N., Kalan, S. M. M., Soltanolkotabi, M., & Avestimehr, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00939 · Cited by 335
On the Optimal Recovery Threshold of Coded Matrix Multiplication
Dutta, S., Fahim, M., Haddadpour, F., Jeong, H., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2929328 · Cited by 290
Coded Computation Over Heterogeneous Clusters
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006961 · Cited by 269
Self-citationDevice Scheduling with Fast Convergence for Wireless Federated Learning
Shi, W., Zhou, S., & Niu, Z. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9149138 · Cited by 216
Motivating Workers in Federated Learning: A Stackelberg Game Perspective
Sarikaya, Y., & Erçetin, Ö. (2019) · IEEE Networking Letters · doi:10.1109/lnet.2019.2947144 · Cited by 209
Hu, S., Chen, X., Ni, W., Hossain, E., & Wang, X. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3086014 · Cited by 190
A Unified Coding Framework for Distributed Computing with Straggling Servers
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2016.7848828 · Cited by 176
Efficient Parallel Split Learning Over Resource-Constrained Wireless Edge Networks
Lin, Z., Zhu, G., Deng, Y., Chen, X., Gao, Y., Huang, K., & Fang, Y. M. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3359040 · Cited by 197
Handling privacy-sensitive medical data with federated learning: Challenges and future directions
Aouedi, O., Sacco, A., Piamrat, K., & Marchetto, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3185673 · Cited by 165
Gradient Coding From Cyclic MDS Codes and Expander Graphs
Raviv, N., Tamo, I., Tandon, R., & Dimakis, A. G. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3029396 · Cited by 179
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously
Li, S., Yu, M., Yang, C.-S., Avestimehr, A. S., Kannan, S., & Viswanath, P. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174305 · Cited by 160
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147
Provably Secure Federated Learning against Malicious Clients
Cao, X., Jia, J., & Gong, N. Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16849 · Cited by 141
An Exact Quantized Decentralized Gradient Descent Algorithm
Reisizadeh, A., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2932876 · Cited by 138
Self-citationReisizadeh, A., Tziotis, I., Hassani, H., Mokhtari, A., & Pedarsani, R. (2022) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2022.3205475 · Cited by 137
Self-citationA Scalable Framework for Wireless Distributed Computing
Li, S., Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2017.2702605 · Cited by 130
Ng, J. S., Lim, W. Y. B., Luong, N. C., Xiong, Z., Asheralieva, A., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3091684 · Cited by 151
Speeding up distributed machine learning using codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2016) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541478 · Cited by 125
Self-citationReliable Distributed Computing for Metaverse: A Hierarchical Game-Theoretic Approach
Jiang, Y., Kang, J., Niyato, D. T., Ge, X., Xiong, Z., Miao, C., & Shen, X. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3204839 · Cited by 124
A Survey on Resource Management in Joint Communication and Computing-Embedded SAGIN
Chen, Q., Guo, Z., Meng, W., Han, S., Li, C., & Quek, T. Q. S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3421523 · Cited by 147
Coded Sparse Matrix MultiplicationPreprint
Wang, S., Liu, J., & Shroff, N. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03430 · Cited by 116
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication
Mallick, A., Chaudhari, M., Sheth, U., Palanikumar, G., & Joshi, G. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366706 · Cited by 114
Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding
Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437563 · Cited by 111
Towards 6G hyper-connectivity: Vision, challenges, and key enabling technologies
Lee, H., Lee, B., Yang, H., Kim, J., Kim, S., Shin, W., Shim, B., & Poor, H. V. (2023) · Journal of Communications and Networks · doi:10.23919/jcn.2023.000006 · Cited by 111
Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks
Prakash, S., Dhakal, S., Akdeniz, M. R., Yona, Y., Talwar, S., Avestimehr, S., & Himayat, N. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036961 · Cited by 109
The Capacity of Private Computation
Sun, H., & Jafar, S. A. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2888494 · Cited by 102
Near-Optimal Straggler Mitigation for Distributed Gradient Methods
Li, S.-Z., Kalan, S. M. M., Avestimehr, A., & Soltanolkotabi, M. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2018.00137 · Cited by 105
Block-Diagonal and LT Codes for Distributed Computing With Straggling Servers
Severinson, A., Amat, A. G. I., & Rosnes, E. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2877391 · Cited by 93
Private and Secure Distributed Matrix Multiplication With Flexible Communication Load
Aliasgari, M., Simeone, O., & Kliewer, J. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2972166 · Cited by 92
Numerically Stable Polynomially Coded Computing
Fahim, M., & Cadambe, V. R. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849468 · Cited by 91
Private Retrieval, Computing, and Learning: Recent Progress and Future Challenges
Ulukuş, Ş., Avestimehr, S., Gastpar, M., Jafar, S. A., Tandon, R., & Tian, C. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3142358 · Cited by 92
Yu, Q., & Avestimehr, A. S. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174167 · Cited by 89
Stochastic Gradient Coding for Straggler Mitigation in Distributed Learning
Bitar, R., Wootters, M. K., & Rouayheb, S. E. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991361 · Cited by 4
Hierarchical Coding for Distributed Computing
Park, H., Lee, K., Sohn, J.-Y., Suh, C., & Moon, J. J. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437669 · Cited by 82
Self-citationSpeeding Up Distributed Gradient Descent by Utilizing Non-persistent Stragglers
Özfatura, E., Gündüz, D., & Ulukuş, Ş. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849684 · Cited by 82
Sun, Y., Zhou, S., & Niu, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3030889 · Cited by 78
Dhakal, S., Prakash, S., Yona, Y., Talwar, S., & Himayat, N. (2019) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps45667.2019.9024521 · Cited by 77
Accelerating Split Federated Learning Over Wireless Communication Networks
Xu, C., Li, J., Liu, Y., Ling, Y., & Wen, M. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3327372 · Cited by 77
Li, K., Tao, M., & Chen, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2985039 · Cited by 76
Minimizing latency for secure distributed computing
Bitar, R., Parag, P., & Rouayheb, S. E. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8007060 · Cited by 72
Latency Minimization for Mobile Edge Computing Networks
Chen, C.-L., Brinton, C. G., & Aggarwal, V. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3117511 · Cited by 74
Adaptive Hierarchical Federated Learning Over Wireless Networks
Xu, B., Xia, W., Wen, W., Liu, P., Zhao, H., & Zhu, H. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3135541 · Cited by 76
Zhang, Y., Liu, D., Duan, M., Li, L., Chen, X., Ren, A., Tan, Y., & Wang, C. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3237752 · Cited by 73
Cross Subspace Alignment Codes for Coded Distributed Batch Computation
Jia, Z., & Jafar, S. A. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3064827 · Cited by 73
Communicate to Learn at the Edge
Gündüz, D., Kurka, D. B., Jankowski, M., Amiri, M. M., Özfatura, E., & Sreekumar, S. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000394 · Cited by 69
CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning
Schlegel, R., Kumar, S., Rosnes, E., & Amat, A. G. I. (2023) · IEEE Transactions on Communications · doi:10.1109/tcomm.2023.3244243 · Cited by 75
Jia, Z., & Jafar, S. A. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3013152 · Cited by 71
Computation Scheduling for Distributed Machine Learning With Straggling Workers
Amiri, M. M., & Gündüz, D. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2952051 · Cited by 68
Secure Coded Multi-Party Computation for Massive Matrix Operations
Nodehi, H. A., & Maddah-Ali, M. A. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3050853 · Cited by 71
Autonomic machine learning platform
Lee, K. M., Yoo, J., Kim, S., Lee, J.-H., & Hong, J. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.07.003 · Cited by 66
Self-citationYu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2017.8262778 · Cited by 65
On the Optimality of D2D Coded Caching With Uncoded Cache Placement and One-Shot Delivery
Yapar, Ç., Wan, K., Schaefer, R. F., & Caire, G. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2940671 · Cited by 63
Wireless MapReduce Distributed Computing
Li, F., Chen, J., & Wang, Z. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2924621 · Cited by 65
Distributed Optimization of Collaborative Regions in Large-Scale Inhomogeneous Fog Computing
Lyu, X., Ren, C., Ni, W., Tian, H., & Liu, R. P. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2815359 · Cited by 62
Timely Distributed Computation With Stragglers
Buyukates, B., & Ulukuş, Ş. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3001873 · Cited by 61
Dhakal, S., Saurav, P., Yona, Y., Talwar, S., & Nageen, H. (2019) · IEEE Conference Proceedings · Cited by 60
Analog Lagrange Coded Computing
Soleymani, M., Mahdavifar, H., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3056377 · Cited by 58
Minimizing Latency for Secure Coded Computing Using Secret Sharing via Staircase Codes
Bitar, R., Parag, P., & Rouayheb, S. E. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2988506 · Cited by 57
CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework
Zhang, Y., Duan, M., Liu, D., Li, L., Ren, A., Chen, X., Tan, Y., & Wang, C. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533794 · Cited by 55
OverSketch: Approximate Matrix Multiplication for the Cloud
Gupta, V., Wang, S., Courtade, T. A., & Ramchandran, K. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622139 · Cited by 55
Self-citationGuo, H., Wang, H., Song, T., Yang, H., Lv, Z., Jin, X., Xue, Z., Ma, R., & Guan, H. (2021) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3472883.3486990 · Cited by 54
Anytime coding for distributed computation
Ferdinand, N. S., & Draper, S. C. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852337 · Cited by 54
Universally Decodable Matrices for Distributed Matrix-Vector Multiplication
Ramamoorthy, A., Tang, L., & Vontobel, P. O. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849451 · Cited by 54
A Hierarchical Incentive Design Toward Motivating Participation in Coded Federated Learning
Ng, J. S., Lim, W. Y. B., Xiong, Z., Cao, X., Niyato, D. T., Leung, C., & Kim, D. I. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126057 · Cited by 55
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse GradientsPreprint
Mitra, A., Jaafar, R. H., Pappas, G. J., & Hassani, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07053 · Cited by 53
Li, S., & Avestimehr, S. (2020) · Foundations and Trends® in Communications and Information Theory · doi:10.1561/0100000103 · Cited by 56
Computation Scheduling for Distributed Machine Learning with Straggling Workers
Amiri, M. M., & Gündüz, D. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682911 · Cited by 50
CodedSketch: A Coding Scheme for Distributed Computation of Approximated Matrix Multiplication
Jahani-Nezhad, T., & Maddah-Ali, M. A. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3068165 · Cited by 50
Polynomially Coded Regression: Optimal Straggler Mitigation via Data EncodingPreprint
Li, S., Kalan, S. M. M., Yu, Q., Soltanolkotabi, M., & Avestimehr, A. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09934 · Cited by 50
Robust Gradient Descent via Moment Encoding and LDPC Codes
Maity, R. K., Rawat, A. S., & Mazumdar, A. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849514 · Cited by 65
C3LES: Codes for Coded Computation that Leverage Stragglers
Das, A. B., Tang, L., & Ramamoorthy, A. (2018) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2018.8613321 · Cited by 49
Communication-Aware Scheduling of Serial Tasks for Dispersed Computing
Yang, C.-S., Pedarsani, R., & Avestimehr, A. S. (2019) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2019.2919553 · Cited by 49
Self-citationSecure Distributed Matrix Computation With Discrete Fourier Transform
Mital, N., Ling, C., & Gündüz, D. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3158868 · Cited by 49
A Comprehensive Survey on Training Acceleration for Large Machine Learning Models in IoT
Wang, H., Qu, Z., Zhou, Q., Zhang, H., Luo, B., Xu, W., Guo, S., & Li, R. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3111624 · Cited by 54
Parity models: erasure-coded resilience for prediction serving systems
Kosaian, J., Rashmi, K. V., & Venkataraman, S. (2019) · Proceedings of the 27th ACM Symposium on Operating Systems Principles · doi:10.1145/3341301.3359654 · Cited by 48
Coded Federated Computing in Wireless Networks with Straggling Devices and Imperfect CSI
Ha, S., Zhang, J., Simeone, O., & Kang, J. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849507 · Cited by 48
GCSA Codes With Noise Alignment for Secure Coded Multi-Party Batch Matrix Multiplication
Chen, Z., Jia, Z., Wang, Z., & Jafar, S. A. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3052934 · Cited by 47
Managing Distributed Machine Learning Lifecycle for Healthcare Data in the Cloud
Zeydan, E., Arslan, Ş. S., & Liyanage, M. (2024) · IEEE Access · doi:10.1109/access.2024.3443520 · Cited by 56
Random Khatri-Rao-Product Codes for Numerically-Stable Distributed Matrix Multiplication
Subramaniam, A. M., Heidarzadeh, A., & Narayanan, K. R. (2019) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2019.8919859 · Cited by 47
Distributed Learning for Wireless Communications: Methods, Applications and Challenges
Qian, L., Yang, P., Xiao, M., Dobre, O. A., Renzo, M. D., Li, J., Han, Z., Qin, Y., & Zhao, J. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3156756 · Cited by 47
Coded computation for multicore setups
Lee, K., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006962 · Cited by 45
Self-citationStraggler-Aware Distributed Learning: Communication–Computation Latency Trade-Off
Özfatura, E., Ulukuş, Ş., & Gündüz, D. (2020) · Entropy · doi:10.3390/e22050544 · Cited by 45
Aktaş, M. F., & Soljanin, E. (2019) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2019.2946464 · Cited by 44
Ali, S., Wang, J., & Leung, V. C. M. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102922 · Cited by 64
Coded Distributed Computing: Straggling Servers and Multistage Dataflows
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852225 · Cited by 43
Distributed Gradient Descent with Coded Partial Gradient Computations
Özfatura, E., Ulukuş, Ş., & Gündüz, D. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683267 · Cited by 43
Fundamental Limits of Approximate Gradient Coding
Wang, S., Liu, J., & Shroff, N. B. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366700 · Cited by 43
Storage, Computation, and Communication: A Fundamental Tradeoff in Distributed Computing
Yan, Q., Yang, S., & Wigger, M. (2018) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2018.8613519 · Cited by 43
Boomerang: Redundancy Improves Latency and Throughput in Payment-Channel Networks
Bagaria, V., Neu, J., & Tse, D. N. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-51280-4_17 · Cited by 43
A Double Auction Mechanism for Resource Allocation in Coded Vehicular Edge Computing
Ng, J. S., Lim, W. Y. B., Xiong, Z., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3131395 · Cited by 43
Gradient Coding with Clustering and Multi-Message Communication
Özfatura, E., Gündüz, D., & Ulukuş, Ş. (2019) · IEEE Data Science Workshop (DSW) · doi:10.1109/dsw.2019.8755563 · Cited by 42
FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction
Damaskinos, G., Guerraoui, R., Kermarrec, A.-M., Nitu, V., Patra, R., & Taı̈ani, F. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3527621 · Cited by 43
Metadata
- Authors
- Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- University of California, Berkeley (US)
- University of California, Santa Barbara (US)
- University of Wisconsin–Madison (US)
- Venue
- IEEE Transactions on Information Theory
- Published
- 2018-03
- Volume
- 64
- Issue
- 3
- Pages
- 1514-1529
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2268702383
- Cited by
- 829
- References
- 122
- Citations by year
- 2026: 24 · 2025: 56 · 2024: 79 · 2023: 101 · 2022: 111 · 2021: 148 · 2020: 130 · 2019: 105 · 2018: 48
- FWCI
- 54.46
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Information Theory (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute for Information & communications Technology Promotion(IITP) grant
- Korea government(MSIT) (2017-0-00694)
- Brain Korea 21 Plus Project, and NSF CIF grant (1703678)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- distributed learning, coded computing, straggler nodes, distributed matrix multiplication, data shuffling, communication bottleneck, multicast, caching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
