The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Speeding Up Distributed Machine Learning Using Codes
Open accessAbstract
Codes are widely used in many engineering applications to offerrobustnessagainstnoise. In large-scale systems, there are several types of noise that can affect the performance of distributed machine learning algorithms—straggler nodes, system failures, or communication bottlenecks—but there has been little interaction cutting across codes, machine learning, and distributed systems. In this paper, we provide theoretical insights on howcodedsolutions can achieve significant gains compared with uncoded ones. We focus on two of the most basic building blocks of distributed learning algorithms:matrix multiplicationanddata shuffling. For matrix multiplication, we use codes to alleviate the effect of stragglers and show that if the number of homogeneous workers is$n$, and the runtime of each subtask has an exponential tail, coded computation can speed up distributed matrix multiplication by a factor of$\log n$. For data shuffling, we use codes to reduce communication bottlenecks, exploiting the excess in storage. We show that when a constant fraction$\alpha $of the data matrix can be cached at each worker, and$n$is the number of workers,coded shufflingreduces the communication cost by a factor of$\left({\alpha + \frac {1}{n}}\right)\gamma (n)$compared with uncoded shuffling, where$\gamma (n)$is the ratio of the cost of unicasting$n$messages to$n$users to multicasting a common message (of the same size) to$n$users. For instance,$\gamma (n) \simeq n$if multicasting a message to$n$users is as cheap as unicasting a message to one user. We also provide experimental results, corroborating our theoretical gains of the coded algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Data poisoning attacks may occur in two ways such as clean-label attack (adversary can poison correct class of data sample) and dirty-label attack (adversary try to misclassify the target label of training dataset) [147], [148].”
From Challenges, Applications and Design Aspects of Federated Learning: A Survey (2021) · cites this paper as [148]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,917
Smith, D. K., & Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 12,385
(2010) · Cited by 4,411
Flagged first, then by citations
Research works printed but not linked: 3
- 10.1109/ISIT.2016.7541478 · DOI
- 2015, StarCluster
- 2015, Open MPI Open Source High Performance Computing
Cited by
Sorted by citations
Items 101–200 of 827
Ramamoorthy, A., Das, A. B., & Tang, L. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2974149 · Cited by 41
Distributed Linearly Separable Computation
Wan, K., Sun, H., Ji, M., & Caire, G. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3127910 · Cited by 41
Slow and Stale Gradients Can Win the Race
Dutta, S., Wang, J., & Joshi, G. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3103770 · Cited by 40
Timely-Throughput Optimal Coded Computing over Cloud Networks
Yang, C.-S., Pedarsani, R., & Avestimehr, A. S. (2019) · ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc) · doi:10.1145/3323679.3326528 · Cited by 40
Self-citationBerrut Approximated Coded Computing: Straggler Resistance Beyond Polynomial Computing
Jahani-Nezhad, T., & Maddah-Ali, M. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3151434 · Cited by 40
OverSketched Newton: Fast Convex Optimization for Serverless Systems
Gupta, V., Kadhe, S., Courtade, T. A., Mahoney, M. W., & Ramchandran, K. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378289 · Cited by 39
Self-citationNumerically Stable Coded Matrix Computations via Circulant and Rotation Matrix Embeddings
Ramamoorthy, A., & Tang, L. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3137266 · Cited by 39
On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems
Wang, B., Xie, J., Lu, K., Wan, Y., & Fu, S. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3095040 · Cited by 39
On Large-Cohort Training for Federated LearningPreprint
Charles, Z., Garrett, Z., Huo, Z., Shmulyian, S., & Smith, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07820 · Cited by 50
Coded Sparse Matrix Computation Schemes That Leverage Partial Stragglers
Das, A. B., & Ramamoorthy, A. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3152827 · Cited by 38
On Model Coding for Distributed Inference and Transmission in Mobile Edge Computing Systems
Zhang, J., & Simeone, O. (2019) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2911496 · Cited by 37
Optimal Load Allocation for Coded Distributed Computation in Heterogeneous Clusters
Kim, D., Park, H., & Choi, J. K. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3030667 · Cited by 37
Distributed Matrix-Vector Multiplication: A Convolutional Coding Approach
Das, A. B., & Ramamoorthy, A. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849395 · Cited by 36
Coded Computing for Distributed Graph Analytics
Prakash, S., Reisizadeh, A., Pedarsani, R., & Avestimehr, S. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437860 · Cited by 36
Self-citationEfficient and Robust Distributed Matrix Computations via Convolutional Coding
Das, A. B., Ramamoorthy, A., & Vaswani, N. (2021) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit45174.2021.9518108 · Cited by 36
Efficient Device Scheduling with Multi-Job Federated Learning
Zhou, C., Liu, J., Jia, J., Zhou, J., Yang, Z., Dai, H., & Dou, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21235 · Cited by 38
Multi-Job Intelligent Scheduling With Cross-Device Federated Learning
Liu, J., Jia, J., Ma, B., Zhou, C., Zhou, J., Zhou, Y., Dai, H., & Dou, D. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3224941 · Cited by 36
Distributed and Private Coded Matrix Computation with Flexible Communication Load
Aliasgari, M., Simeone, O., & Kliewer, J. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849606 · Cited by 35
Fast and Efficient Distributed Matrix-vector Multiplication Using Rateless Fountain Codes
Mallick, A., Chaudhari, M., & Joshi, G. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682347 · Cited by 35
Hierarchical Coded Matrix Multiplication
Kianidehkordi, S., Ferdinand, N., & Draper, S. C. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3036763 · Cited by 35
Gu, S., Zhang, Q., & Xiang, W. (2020) · IEEE Wireless Communications · doi:10.1109/mwc.001.2000108 · Cited by 35
X-Secure T-Private Federated Submodel Learning With Elastic Dropout Resilience
Jia, Z., & Jafar, S. A. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3165400 · Cited by 35
Information Theoretic Limits of Data Shuffling for Distributed Learning
Attia, M. A., & Tandon, R. (2016) · Globecom · doi:10.1109/glocom.2016.7841903 · Cited by 34
Yang, Y. L., Interlandi, M., Grover, P., Kar, S., Amizadeh, S., & Weimer, M. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849212 · Cited by 34
Yang, C.-S., Pedarsani, R., & Avestimehr, A. S. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849235 · Cited by 34
Self-citationZhang, J., & Simeone, O. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979762 · Cited by 34
Damaskinos, G., Guerraoui, R., Kermarrec, A.-M., Nitu, V., Patra, R., & Taı̈ani, F. (2020) · International Middleware Conference · doi:10.1145/3423211.3425685 · Cited by 37
Fault- Tolerant Dot-Product Engines
Roth, R. M. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437919 · Cited by 34
Matrix sparsification for coded matrix multiplication
Suh, G., Lee, K., & Suh, C. (2017) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2017.8262883 · Cited by 33
Self-citationNear Optimal Coded Data Shuffling for Distributed Learning
Attia, M. A., & Tandon, R. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2926704 · Cited by 33
A Node-Selection-Based Sub-Task Assignment Method for Coded Edge Computing
Zhao, S. (2019) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2909034 · Cited by 33
Gradient Coding Based on Block Designs for Mitigating Adversarial Stragglers
Kadhe, S., Koyluoglu, O. O., & Ramchandran, K. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849690 · Cited by 33
Self-citationCoded Distributed Computing with Node Cooperation Substantially Increases Speedup Factors
Parrinello, E., Lampiris, E., & Elia, P. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437333 · Cited by 33
Compressed Coded Distributed Computing
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437882 · Cited by 32
A Pliable Index Coding Approach to Data Shuffling
Song, L., Fragouli, C., & Zhao, T. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2954338 · Cited by 32
Latency Optimization for Coded Computation Straggled by Wireless Transmission
Wu, F., & Chen, L. (2020) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2020.2983359 · Cited by 32
Coded Computing for Resilient, Secure, and Privacy-Preserving Distributed Matrix Multiplication
Yu, Q., & Avestimehr, A. S. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3032196 · Cited by 33
Joint Coding and Scheduling Optimization for Distributed Learning Over Wireless Edge Networks
Huynh, N. V., Hoang, D. T., Nguyen, D. N., & Dutkiewicz, E. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118432 · Cited by 32
On Delay-Optimal Scheduling in Queueing Systems with ReplicationsPreprint
Sun, Y. J., Koksal, C. E., & Shroff, N. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07322 · Cited by 32
Sheth, U., Dutta, S., Chaudhari, M., Jeong, H., Yang, Y. L., Kohonen, J., Roos, T., & Grover, P. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622429 · Cited by 31
Improved Latency-communication Trade-off for Map-shuffle-reduce Systems with Stragglers
Zhang, J., & Simeone, O. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682404 · Cited by 31
Live Gradient Compensation for Evading Stragglers in Distributed Learning
Xu, J., Huang, S., Song, L., & Lan, T. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488815 · Cited by 34
Fundamental Limits of Coded Linear TransformPreprint
Wang, S., Liu, J., Shroff, N. B., & Yang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09791 · Cited by 31
MOB-FL: Mobility-Aware Federated Learning for Intelligent Connected Vehicles
Xie, B., Sun, Y., Zhou, S., Niu, Z., Xu, Y., Chen, J., & Gündüz, D. (2023) · International Conference on Communications · doi:10.1109/icc45041.2023.10279773 · Cited by 30
Private Coded Matrix Multiplication
Kim, M., Yang, H., & Lee, J. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2940895 · Cited by 30
Optimizing Age of Information and Security of the Next-Generation Internet of Everything Systems
Asheralieva, A., & Niyato, D. T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3173912 · Cited by 30
Adaptive Verifiable Coded Computing: Towards Fast, Secure and Private Distributed Machine Learning
Tang, T., Ali, R. E., Hashemi, H., Gangwani, T., Avestimehr, S., & Annavaram, M. (2022) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps53621.2022.00067 · Cited by 36
CodedReduce: A Fast and Robust Framework for Gradient Aggregation in Distributed Learning
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3109097 · Cited by 30
Self-citationSAZD: A Low Computational Load Coded Distributed Computing Framework for IoT Systems
Dai, M., Zheng, Z., Zhang, S., Wang, H., & Lin, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2974045 · Cited by 30
Robust Gradient Descent via Moment Encoding with LDPC CodesPreprint
Maity, R. K., Rawat, A. S., & Mazumdar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08327 · Cited by 30
Stochastic Learning Under Random Reshuffling With Constant Step-Sizes
Ying, B., Yuan, K., Vlaski, S., & Sayed, A. H. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2878551 · Cited by 29
Factored LT and Factored Raptor Codes for Large-Scale Distributed Matrix Multiplication
Pradhan, A. K., Heidarzadeh, A., & Narayanan, K. R. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174314 · Cited by 29
Data Encoding for Byzantine-Resilient Distributed Optimization
Data, D., Song, L., & Diggavi, S. N. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3035868 · Cited by 29
Coded Matrix Multiplication on a Group-Based Model
Kim, M., Sohn, J.-Y., & Moon, J. J. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849317 · Cited by 28
Learning-Based Coded Computation
Kosaian, J., Rashmi, K. V., & Venkataraman, S. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2983165 · Cited by 29
Age-Based Coded Computation for Bias Reduction in Distributed Learning
Özfatura, E., Buyukates, B., Gündüz, D., & Ulukuş, Ş. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322412 · Cited by 28
List-Decodable Coded Computing: Breaking the Adversarial Toleration Barrier
Soleymani, M., Ali, R. E., Mahdavifar, H., & Avestimehr, A. S. (2021) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit45174.2021.9518044 · Cited by 27
Staircase-PIR: Universally Robust Private Information Retrieval
Bitar, R., & Rouayheb, S. E. (2018) · 2018 IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2018.8613532 · Cited by 27
Private Secure Coded Computation
Kim, M., & Lee, J. (2019) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2934436 · Cited by 27
Heterogeneity-aware Gradient Coding for Straggler Tolerance
Wang, H., Guo, S., Tang, B., Li, R., & Li, C. (2019) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00062 · Cited by 27
Kim, K. T., Joe‐Wong, C., & Chiang, M. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155226 · Cited by 27
Optimal Task Allocation and Coding Design for Secure Edge Computing With Heterogeneous Edge Devices
Wang, J., Cao, C., Wang, J., Lu, K., Jukan, A., & Zhao, W. (2021) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2021.3050012 · Cited by 27
Asheralieva, A., Niyato, D. T., & Xiong, Z. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3097380 · Cited by 27
Privacy-Preserving Coded Mobile Edge Computing for Low-Latency Distributed Inference
Schlegel, R., Kumar, S., Rosnes, E., & Amat, A. G. I. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3142295 · Cited by 27
A Survey on Trusted Distributed Artificial Intelligence
Ağca, M. A., Faye, S., & Khadraoui, D. (2022) · IEEE Access · doi:10.1109/access.2022.3176385 · Cited by 30
A New Combinatorial Coded Design for Heterogeneous Distributed Computing
Woolsey, N., Chen, R., & Ji, M. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3087628 · Cited by 27
Coded distributed computing: Fundamental limits and practical challenges
Li, S., Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2016.7869092 · Cited by 26
Distributed Fog Computing Based on Batched Sparse Codes for Industrial Control
Yue, J., Xiao, M., & Pang, Z. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2857203 · Cited by 26
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849431 · Cited by 26
Self-citationRateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication
Mallick, A., Chaudhari, M., Sheth, U., Palanikumar, G., & Joshi, G. (2020) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/3393691.3394214 · Cited by 26
Adaptive Private Distributed Matrix Multiplication
Bitar, R., Xhemrishi, M., & Wachter-Zeh, A. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3143199 · Cited by 26
Improved Constructions for Secure Multi-Party Batch Matrix Multiplication
Zhu, J., Yan, Q., & Tang, X. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3107942 · Cited by 26
Addressing Unreliability in Emerging Devices and Non-von Neumann Architectures Using Coded Computing
Dutta, S., Jeong, H., Yang, Y. L., Cadambe, V. R., Low, T. M., & Grover, P. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2986362 · Cited by 25
Fundamental Limits of Decentralized Data Shuffling
Wan, K., Tuninetti, D., Ji, M., Caire, G., & Piantanida, P. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.2966197 · Cited by 25
Service Rate Region: A New Aspect of Coded Distributed System Design
Aktaş, M. F., Joshi, G., Kadhe, S., Kazemi, F., & Soljanin, E. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3117695 · Cited by 25
Coded Computing and Cooperative Transmission for Wireless Distributed Matrix Multiplication
Li, K., Tao, M., Zhang, J., & Simeone, O. (2021) · IEEE Transactions on Communications · doi:10.1109/tcomm.2021.3053018 · Cited by 25
Fast and Secure Computational Offloading With Lagrange Coded Mobile Edge Computing
Asheralieva, A., & Niyato, D. T. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3070723 · Cited by 25
A Fundamental Tradeoff between Computation and Communication in Distributed ComputingPreprint
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.07086 · Cited by 25
A sequential approximation framework for coded distributed optimization
Zhu, J., Pu, Y., Gupta, V., Tomlin, C. J., & Ramchandran, K. (2017) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2017.8262879 · Cited by 22
Self-citationHierarchical Coded Gradient Aggregation for Learning at the Edge
Prakash, S., Reisizadeh, A., Pedarsani, R., & Avestimehr, A. S. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174077 · Cited by 23
Self-citationPrivate and Secure Distributed Matrix Multiplication Schemes for Replicated or MDS-Coded Servers
Li, J., & Hollanti, C. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3147638 · Cited by 24
Coded Wireless Distributed Computing With Packet Losses and Retransmissions
Han, D.-J., Sohn, J.-Y., & Moon, J. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3091465 · Cited by 22
Yan, Q., Yang, S., & Wigger, M. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3158828 · Cited by 23
Distributed Stochastic Gradient Descent Using LDGM Codes
Horii, S., Yoshida, T., Kobayashi, M., & Matsushima, T. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849580 · Cited by 23
Safety Design of a Convolutional Neural Network Accelerator with Error Localization and Correction
Xu, Z. G., & Abraham, J. A. (2019) · IEEE International Test Conference (ITC) · doi:10.1109/itc44170.2019.9000149 · Cited by 23
Edge Computing in the Dark: Leveraging Contextual-Combinatorial Bandit and Coded Computing
Yang, C.-S., Pedarsani, R., & Avestimehr, A. S. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3058685 · Cited by 26
Self-citationFedPCC: Parallelism of Communication and Computation for Federated Learning in Wireless Networks
Zhang, H., Tian, H., Dong, M., Ota, K., & Jia, J. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3170471 · Cited by 23
Multi-User Linearly-Separable Distributed Computing
Khalesi, A., & Elia, P. (2023) · IEEE Transactions on Information Theory · doi:10.1109/tit.2023.3283967 · Cited by 23
A Fundamental Storage-Communication Tradeoff in Distributed Computing with Straggling Nodes
Yan, Q., Wigger, M., Yang, S., & Tang, X. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849615 · Cited by 23
Kim, N., Kim, D., Lee, J., Niyato, D. T., & Choi, J. K. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3045277 · Cited by 23
Election Coding for Distributed Learning: Protecting SignSGD against Byzantine AttacksPreprint
Sohn, J.-Y., Han, D.-J., Choi, B., & Moon, J. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.06093 · Cited by 23
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers
Ferdinand, N. S., & Draper, S. C. (2018) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2018.8635903 · Cited by 22
Fundamental Limits of Approximate Gradient Coding
Wang, S., Liu, J., & Shroff, N. B. (2020) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/3393691.3394188 · Cited by 22
Coding for Distributed Fog Computing in Internet of Mobile Things
Yue, J., & Xiao, M. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2963668 · Cited by 22
Cao, H., Yan, Q., Tang, X., & Han, G. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3122873 · Cited by 22
Stochastic Coded Offloading Scheme for Unmanned-Aerial-Vehicle-Assisted Edge Computing
Ng, W. C., Lim, W. Y. B., Xiong, Z., Niyato, D. T., Miao, C., Han, Z., & Kim, D. I. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3150472 · Cited by 23
Li, P., Cheng, G., Huang, X., Kang, J., Yu, R., Wu, Y., Pan, M., & Niyato, D. T. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3245601 · Cited by 22
Rateless codes for near-perfect load balancing in distributed matrix-vector multiplication
Mallick, A., Chaudhari, M., Sheth, U., Palanikumar, G., & Joshi, G. (2022) · Communications of the ACM · doi:10.1145/3524298 · Cited by 22
A Survey of Coded Distributed ComputingPreprint
Ng, J. S., Lim, W. Y. B., Luong, N. C., Xiong, Z., Asheralieva, A., Niyato, D. T., Leung, C., & Miao, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.09048 · Cited by 22
Jiang, Y., Kang, J., Niyato, D. T., Ge, X., Xiong, Z., & Miao, C. (2021) · arXiv (Cornell University) · Cited by 22
Metadata
- Authors
- Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- University of California, Berkeley (US)
- University of California, Santa Barbara (US)
- University of Wisconsin–Madison (US)
- Venue
- IEEE Transactions on Information Theory
- Published
- 2018-03
- Volume
- 64
- Issue
- 3
- Pages
- 1514-1529
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2268702383
- Cited by
- 829
- References
- 122
- Citations by year
- 2026: 24 · 2025: 56 · 2024: 79 · 2023: 101 · 2022: 111 · 2021: 148 · 2020: 130 · 2019: 105 · 2018: 48
- FWCI
- 54.46
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Information Theory (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute for Information & communications Technology Promotion(IITP) grant
- Korea government(MSIT) (2017-0-00694)
- Brain Korea 21 Plus Project, and NSF CIF grant (1703678)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- distributed learning, coded computing, straggler nodes, distributed matrix multiplication, data shuffling, communication bottleneck, multicast, caching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
