Speeding Up Distributed Machine Learning Using Codes
Open accessAbstract
Codes are widely used in many engineering applications to offerrobustnessagainstnoise. In large-scale systems, there are several types of noise that can affect the performance of distributed machine learning algorithms—straggler nodes, system failures, or communication bottlenecks—but there has been little interaction cutting across codes, machine learning, and distributed systems. In this paper, we provide theoretical insights on howcodedsolutions can achieve significant gains compared with uncoded ones. We focus on two of the most basic building blocks of distributed learning algorithms:matrix multiplicationanddata shuffling. For matrix multiplication, we use codes to alleviate the effect of stragglers and show that if the number of homogeneous workers is$n$, and the runtime of each subtask has an exponential tail, coded computation can speed up distributed matrix multiplication by a factor of$\log n$. For data shuffling, we use codes to reduce communication bottlenecks, exploiting the excess in storage. We show that when a constant fraction$\alpha $of the data matrix can be cached at each worker, and$n$is the number of workers,coded shufflingreduces the communication cost by a factor of$\left({\alpha + \frac {1}{n}}\right)\gamma (n)$compared with uncoded shuffling, where$\gamma (n)$is the ratio of the cost of unicasting$n$messages to$n$users to multicasting a common message (of the same size) to$n$users. For instance,$\gamma (n) \simeq n$if multicasting a message to$n$users is as cheap as unicasting a message to one user. We also provide experimental results, corroborating our theoretical gains of the coded algorithms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data poisoning attacks may occur as clean-label attacks or dirty-label attacks.
“Data poisoning attacks may occur in two ways such as clean-label attack (adversary can poison correct class of data sample) and dirty-label attack (adversary try to misclassify the target label of training dataset) [147], [148].”
From Challenges, Applications and Design Aspects of Federated Learning: A Survey (2021) · cites this paper as [148]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601 · see citing works
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476 · see citing works
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,863 · see citing works
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,969 · see citing works
(2010) · Cited by 4,411 · see citing works
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719 · see citing works
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,189 · see citing works
Weik, M. H. (2000) · Springer eBooks · doi:10.1007/1-4020-0613-6_12436 · Cited by 2,584 · see citing works
Elements of Information Theory
Nalewajski, R. F. (2011) · Springer eBooks · doi:10.1007/978-3-642-20180-6_8 · Cited by 2,310 · see citing works
Stochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_25 · Cited by 2,097 · see citing works
Network Coding for Distributed Storage Systems
Dimakis, A. G., Godfrey, P. B., Wu, Y., Wainwright, M. J., & Ramchandran, K. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2054295 · Cited by 1,913 · see citing works
Self-citationMLlib: Machine Learning in Apache SparkPreprint
Meng, X., Bradley, J. K., Yavuz, B., Sparks, E., Venkataraman, S., Liu, D., Freeman, J., Tsai, D., Amde, M., Owen, S., et al. (2015) · Journal of machine learning research · doi:10.48550/arxiv.1505.06807 · Cited by 1,898 · see citing works
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852 · see citing works
Maddah-Ali, M. A., & Niesen, U. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2306938 · Cited by 1,726 · see citing works
Low, Y., Bickson, D., Gonzalez, J. E., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2212351.2212354 · Cited by 1,656 · see citing works
Improving MapReduce performance in heterogeneous environments
Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., & Stoica, I. G. (2008) · doi:10.5555/1855741.1855744 · Cited by 1,625 · see citing works
Costello, D. J., & Shu, L. (2004) · Cited by 1,564 · see citing works
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211 · see citing works
Scaling distributed machine learning with the parameter server
Li, M., Andersen, D. G., Park, J. W., Smola, A. J., Ahmed, A. A., Josifovski, V., Long, J. E., Shekita, E. J., & Su, B.-Y. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685095 · Cited by 1,178 · see citing works
Parallelized Stochastic Gradient Descent
Zinkevich, M., Weimer, M., Li, L., & Smola, A. J. (2010) · Cited by 1,124 · see citing works
Scaling Distributed Machine Learning with the Parameter Server
Li, M. (2014) · International Conference on Big Data Science and Computing (BigDataScience) · doi:10.1145/2640087.2644155 · Cited by 1,000 · see citing works
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2012) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2011.2161027 · Cited by 947 · see citing works
Erasure coding in windows azure storage
Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., & Yekhanin, S. (2012) · Cited by 928 · see citing works
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2010) · Cited by 884 · see citing works
On the Locality of Codeword Symbols
Gopalan, P., Huang, C., Simitci, H., & Yekhanin, S. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2012.2208937 · Cited by 836 · see citing works
Rashmi, K. V., Shah, N. B., & Kumar, P. V. (2011) · NOT FOUND REPOSITORY (Indian Institute of Science Bangalore) · Cited by 701 · see citing works
Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., Dimakis, A. G., Vadali, R., Chen, S., & Borthakur, D. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2535573.2488339 · Cited by 690 · see citing works
Self-citationDiffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks
Chen, J., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2198470 · Cited by 673 · see citing works
Decentralized Coded Caching Attains Order-Optimal Memory-Rate Tradeoff
Maddah-Ali, M. A., & Niesen, U. (2014) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2014.2317316 · Cited by 667 · see citing works
Reining in the outliers in map-reduce clusters using Mantri
Ananthanarayanan, G., Kandula, S., Greenberg, A. G., Stoica, I. G., Lu, Y., Saha, B., & Harris, E. F. (2010) · Operating Systems Design and Implementation · doi:10.5555/1924943.1924962 · Cited by 660 · see citing works
Rashmi, K. V., Shah, N. B., & Kumar, P. V. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2159049 · Cited by 589 · see citing works
Index Coding With Side Information
Bar-Yossef, Z., Birk, Y., Jayram, T. S., & Kol, T. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2103753 · Cited by 474 · see citing works
Effective straggler mitigation: attack of the clones
Ananthanarayanan, G., Ghodsi, A., Shenker, S. J., & Stoica, I. G. (2013) · Cited by 422 · see citing works
Communication-optimal Parallel and Sequential QR and LU Factorizations
Demmel, J. W., Grigori, L., Hoemmen, M. F., & Julien Langou, F. (2012) · SIAM Journal on Scientific Computing · doi:10.1137/080731992 · Cited by 407 · see citing works
Parallel stochastic gradient algorithms for large-scale matrix completion
Recht, B., & Ré, C. (2013) · Mathematical Programming Computation · doi:10.1007/s12532-013-0053-8 · Cited by 378 · see citing works
“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products
Dutta, S., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2927558 · Cited by 362 · see citing works
Birk, Y., & Kol, T. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.874540 · Cited by 338 · see citing works
Distributed delayed stochastic optimization
Agarwal, A., & Duchi, J. C. (2012) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426626 · Cited by 318 · see citing works
Augmenting data center networks with multi-gigabit wireless links
Halperin, D., Kandula, S., Padhye, J. D., Bahl, P., & Wetherall, D. J. (2011) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2018436.2018442 · Cited by 312 · see citing works
MLbase: A Distributed Machine-learning System
Kraska, T., Talwalkar, A., & Duchi, J. C. (2013) · Cited by 310 · see citing works
Optimal linear codes with a local-error-correction property
Prakash, N., Kamath, G. M., Lalitha, V., & Kumar, P. V. (2012) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2012.6284028 · Cited by 304 · see citing works
Huang, C., Chen, M., & Li, J. (2007) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2007.37 · Cited by 300 · see citing works
Bekkerman, R., Bilenko, M., & Langford, J. C. (2011) · doi:10.1145/2107736.2107740 · Cited by 290 · see citing works
Coded Computation Over Heterogeneous Clusters
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006961 · Cited by 269 · see citing works
Self-citationRashmi, K. V., Shah, N. B., Gu, D., Kuang, H., Borthakur, D., & Ramchandran, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.0186 · Cited by 266 · see citing works
Self-citationSelf-repairing homomorphic codes for distributed storage systems
Oggier, F., & Datta, A. (2011) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2011.5934901 · Cited by 261 · see citing works
High-dimensional coded matrix multiplication
Lee, K., Suh, C., & Ramchandran, K. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006963 · Cited by 248 · see citing works
Self-citation1Optimal Locally Repairable and Secure Codes for Distributed Storage SystemsPreprint
Rawat, A. S., Koyluoglu, O. O., Silberstein, N., & Vishwanath, S. (2016) · Cited by 224 · see citing works
Papailiopoulos, D., & Dimakis, A. G. (2012) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2012.6284027 · Cited by 212 · see citing works
Self-citationA "hitchhiker's" guide to fast and efficient data reconstruction in erasure-coded data centers
Rashmi, K. V., Shah, N. B., Gu, D., Kuang, H., Borthakur, D., & Ramchandran, K. (2014) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2619239.2626325 · Cited by 209 · see citing works
Self-citationOptimal Locally Repairable and Secure Codes for Distributed Storage Systems
Rawat, A. S., Koyluoglu, O. O., Silberstein, N., & Vishwanath, S. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2013.2288784 · Cited by 200 · see citing works
Exact-Repair MDS Code Construction Using Interference Alignment
Suh, C., & Ramchandran, K. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2105003 · Cited by 200 · see citing works
Self-citationSimple regenerating codes: Network coding for cloud storage
Papailiopoulos, D., Luo, J., Dimakis, A. G., Huang, C., & Li, J. (2012) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2012.6195703 · Cited by 197 · see citing works
Self-citationBekkerman, R., Bilenko, M., & Langford, J. C. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139042918 · Cited by 193 · see citing works
On the Delay-Storage Trade-Off in Content Download from Coded Distributed Storage Systems
Joshi, G., Liu, Y., & Soljanin, E. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.140518 · Cited by 191 · see citing works
A Unified Coding Framework for Distributed Computing with Straggling Servers
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2016.7848828 · Cited by 176 · see citing works
MLI: An API for Distributed Machine Learning
Sparks, E., Talwalkar, A., Smith, V., Kottalam, J., Pan, X., Gonzalez, J. E., Franklin, M. J. M., Jordan, M. I., & Kraska, T. (2013) · Proceedings · doi:10.1109/icdm.2013.158 · Cited by 151 · see citing works
Augmenting data center networks with multi-gigabit wireless links
Halperin, D., Kandula, S., Padhye, J. D., Bahl, P., & Wetherall, D. J. (2011) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2043164.2018442 · Cited by 137 · see citing works
Reducing Latency via Redundant Requests
Gardner, K., Zbarsky, S., Doroudi, S., Harchol‐Balter, M., & Esa Hyytiä, A. (2015) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/2745844.2745873 · Cited by 131 · see citing works
Efficient Redundancy Techniques for Latency Reduction in Cloud Systems
Joshi, G., Soljanin, E., & Wornell, G. W. (2017) · ACM Transactions on Modeling and Performance Evaluation of Computing Systems · doi:10.1145/3055281 · Cited by 130 · see citing works
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447112 · Cited by 128 · see citing works
Speeding up distributed machine learning using codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2016) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541478 · Cited by 125 · see citing works
Self-citationZhang, C., & Ré, C. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732977.2733001 · Cited by 121 · see citing works
Distributed Delayed Stochastic OptimizationPreprint
Agarwal, A., & Duchi, J. C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1104.5525 · Cited by 117 · see citing works
Why random reshuffling beats stochastic gradient descent
Gürbüzbalaban, M., Ozdaglar, A., & Parrilo, P. A. (2019) · Mathematical Programming · doi:10.1007/s10107-019-01440-w · Cited by 110 · see citing works
XORing Elephants: Novel Erasure Codes for Big DataPreprint
Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., Dimakis, A. G., Vadali, R., Chen, S., & Borthakur, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3791 · Cited by 102 · see citing works
Self-citationCodes can reduce queueing delay in data centers
Huang, L., Pawar, S., Zhang, H., & Ramchandran, K. (2012) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2012.6284026 · Cited by 99 · see citing works
Self-citationA "hitchhiker's" guide to fast and efficient data reconstruction in erasure-coded data centers
Rashmi, K. V., Shah, N. B., Gu, D., Kuang, H., Borthakur, D., & Ramchandran, K. (2014) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2740070.2626325 · Cited by 87 · see citing works
Self-citationEfficient task replication for fast response times in parallel computation
Wang, D., Joshi, G., & Wornell, G. W. (2014) · doi:10.1145/2591971.2592042 · Cited by 86 · see citing works
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2015) · Cited by 85 · see citing works
Repair optimal erasure codes through hadamard designs
Papailiopoulos, D., Dimakis, A. G., & Cadambe, V. R. (2011) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2011.6120328 · Cited by 84 · see citing works
Self-citationFundamental limits of distributed caching in D2D wireless networks
Ji, M., Caire, G., & Molisch, A. F. (2013) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2013.6691247 · Cited by 82 · see citing works
Reducing Latency via Redundant Requests
Gardner, K., Zbarsky, S., Doroudi, S., Harchol‐Balter, M., & Esa Hyytiä, A. (2015) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2796314.2745873 · Cited by 82 · see citing works
Karamchandani, N., Niesen, U., Maddah-Ali, M. A., & Diggavi, S. N. (2014) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2014.6875212 · Cited by 78 · see citing works
BLAS (Basic Linear Algebra Subprograms).
Geijn, R. A., & Goto, K. (2011) · Cited by 77 · see citing works
Zhu, Y., Zhou, X., Zhang, Z., Zhou, L., Vahdat, A. M., Zhao, B. Y., & Zheng, H.-T. (2014) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/2639108.2639140 · Cited by 75 · see citing works
TOFEC: Achieving optimal throughput-delay trade-off of cloud storage using erasure codes
Liang, G., & Kozat, U. C. (2014) · doi:10.1109/infocom.2014.6848010 · Cited by 75 · see citing works
Reliable Memories with Subline Accesses
Han, J., & Lastras-Montaño, L. A. (2007) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2007.4557599 · Cited by 73 · see citing works
The MDS queue: Analysing the latency performance of erasure codes
Shah, N. B., Lee, K., & Ramchandran, K. (2014) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2014.6874955 · Cited by 72 · see citing works
Self-citationMinimizing latency for secure distributed computing
Bitar, R., Parag, P., & Rouayheb, S. E. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8007060 · Cited by 72 · see citing works
When do redundant requests reduce latency ?
Shah, N. B., Lee, K., & Ramchandran, K. (2013) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2013.6736597 · Cited by 70 · see citing works
Self-citationPedarsani, R., Maddah-Ali, M. A., & Niesen, U. (2014) · International Conference on Communications · doi:10.1109/icc.2014.6883597 · Cited by 68 · see citing works
Self-citationError resilience in distributed storage via rank-metric codes
Silberstein, N., Rawat, A. S., & Vishwanath, S. (2012) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2012.6483348 · Cited by 67 · see citing works
Arslan, M. Y., Singh, I., Singh, S. N., Madhyastha, H. V., Sundaresan, K., & Krishnamurthy, S. V. (2012) · ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) · doi:10.1145/2413176.2413199 · Cited by 66 · see citing works
MDS array codes with optimal rebuilding
Tamo, I., Wang, Z., & Bruck, J. (2011) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2011.6033733 · Cited by 65 · see citing works
Kamath, G. M., Prakash, N., Lalitha, V., & Kumar, P. V. (2012) · NOT FOUND REPOSITORY (Indian Institute of Science Bangalore) · doi:10.48550/arxiv.1211.1932 · Cited by 64 · see citing works
Fundamental tradeoff between computation and communication in distributed computing
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541612 · Cited by 63 · see citing works
Anytime coding for distributed computation
Ferdinand, N. S., & Draper, S. C. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852337 · Cited by 54 · see citing works
Optimal Repair of MDS Codes in Distributed Storage via Subspace Interference AlignmentPreprint
Cadambe, V. R., Huang, C., Jafar, S. A., & Li, J. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1106.1250 · Cited by 50 · see citing works
Blake, I. F. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009283403.008 · Cited by 49 · see citing works
The MDS Queue: Analysing the Latency Performance of Erasure Codes
Lee, K., Shah, N. B., Huang, L., & Ramchandran, K. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2674671 · Cited by 49 · see citing works
Self-citationCoded computation for multicore setups
Lee, K., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006962 · Cited by 45 · see citing works
Self-citationKamath, G. M., Prakash, N., Lalitha, V., & Kumar, P. V. (2013) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2013.6620498 · Cited by 44 · see citing works
Gradient CodingPreprint
Tandon, R., Lei, Q., Dimakis, A. G., & Karampatziakis, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03301 · Cited by 41 · see citing works
Communication-avoiding parallel strassen: implementation and performance
Lipshitz, B., Ballard, G., Demmel, J. W., & Schwartz, O. (2012) · doi:10.5555/2388996.2389133 · Cited by 39 · see citing works
Efficient task replication for fast response times in parallel computation
Wang, D., Joshi, G., & Wornell, G. W. (2014) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2637364.2592042 · Cited by 37 · see citing works
Communication-avoiding algorithms for linear algebra and beyond
Demmel, J. W. (2013) · Proceedings - IEEE International Parallel and Distributed Processing Symposium · doi:10.1109/ipdps.2013.123 · Cited by 36 · see citing works
Analyzing the download time of availability codes
Kadhe, S., Soljanin, E., & Sprintson, A. (2015) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2015.7282699 · Cited by 35 · see citing works
Information Theoretic Limits of Data Shuffling for Distributed Learning
Attia, M. A., & Tandon, R. (2016) · Globecom · doi:10.1109/glocom.2016.7841903 · Cited by 34 · see citing works
Provably delay efficient data retrieving in storage clouds
Sun, Y. J., Zheng, Z., Koksal, C. E., Kim, K.-H., & Shroff, N. B. (2015) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2015.7218426 · Cited by 32 · see citing works
MLI: An API for Distributed Machine LearningPreprint
Sparks, E., Talwalkar, A., Smith, V., Kottalam, J., Pan, X., Gonzalez, J. E., Franklin, M. J. M., Jordan, M. I., & Kraska, T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.5426 · Cited by 32 · see citing works
MDS Array Codes with Optimal RebuildingPreprint
Tamo, I., Wang, Z., & Bruck, J. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.3737 · Cited by 30 · see citing works
Fundamental Limits of Distributed Caching in D2D Wireless NetworksPreprint
Ji, M., Caire, G., & Molisch, A. F. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5856 · Cited by 27 · see citing works
A Fundamental Tradeoff between Computation and Communication in Distributed ComputingPreprint
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.07086 · Cited by 25 · see citing works
A pliable index coding approach to data shuffling
Song, L., Fragouli, C., & Zhao, T. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006991 · Cited by 25 · see citing works
Hierarchical Coded CachingPreprint
Karamchandani, N., Niesen, U., Maddah-Ali, M. A., & Diggavi, S. N. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.7007 · Cited by 23 · see citing works
Codes Can Reduce Queueing Delay in Data CentersPreprint
Huang, L., Pawar, S., Zhang, H., et al. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1202.1359 · Cited by 18 · see citing works
On the worst-case communication overhead for distributed data shuffling
Attia, M. A., & Tandon, R. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852338 · Cited by 16 · see citing works
A Unified Coding Framework for Distributed Computing with Straggling ServersPreprint
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.01690 · Cited by 16 · see citing works
On index coding with side information
Dau, S. H. (2012) · doi:10.32657/10356/48642 · Cited by 15 · see citing works
When do the availability codes make the stored data more available?
Kadhe, S., Soljanin, E., & Sprintson, A. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447111 · Cited by 15 · see citing works
DimmWitted: A Study of Main-Memory Statistical AnalyticsPreprint
Zhang, C., & Ré, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.7550 · Cited by 15 · see citing works
Author retrospective for optimizing matrix multiply using PHiPAC
Bilmes, J., Asanović, K., Chin, C., & Demmel, J. (2014) · International Conference on Supercomputing · doi:10.1145/2591635.2591656 · Cited by 13 · see citing works
Replicated Data Placement for Uncertain Scheduling
Chaubey, M., & Saule, É. (2015) · doi:10.1109/ipdpsw.2015.50 · Cited by 12 · see citing works
On scheduling redundant requests with cancellation overheads
Lee, K., Pedarsani, R., & Ramchandran, K. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7446991 · Cited by 12 · see citing works
Self-citationComputing while Charging: Building a Distributed Computing Infrastructure using Smartphones
Bakiadarshani, J. (2017) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2017.3060 · Cited by 6 · see citing works
A Pliable Index Coding Approach to Data ShufflingPreprint
Song, L., Fragouli, C., & Zhao, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.05540 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- K. Lee, M. Lam, R. Pedarsani, D. Papailiopoulos, and K. Ramchandran, "Speeding up distributed machine learning using codes," IEEE Transactions on Information Theory, vol. PP, no. 99, pp. 1-1, 2017.
- --, "Speeding up distributed machine learning using codes," Presented at 2015 Neural Information Processing Systems (NIPS): Workshop on Machine Learning Systems, December 2015.
Cited by
Sorted by citations
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587 · see citing works
Self-citationMachine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548 · see citing works
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488 · see citing works
A Fundamental Tradeoff Between Computation and Communication in Distributed Computing
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2756959 · Cited by 458 · see citing works
Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix MultiplicationPreprint
Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10464 · Cited by 439 · see citing works
Gradient Coding: Avoiding Stragglers in Distributed Learning
Tandon, R., Lei, Q., Dimakis, A. G., & Karampatziakis, N. (2017) · International Conference on Machine Learning · Cited by 431 · see citing works
Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning
Shi, W., Zhou, S., Niu, Z., Jiang, M., & Geng, L. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3025446 · Cited by 381 · see citing works
“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products
Dutta, S., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2927558 · Cited by 362 · see citing works
Lagrange Coded Computing: Optimal Design for Resiliency, Security and PrivacyPreprint
Yu, Q., Li, S., Raviv, N., Kalan, S. M. M., Soltanolkotabi, M., & Avestimehr, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00939 · Cited by 325 · see citing works
On the Optimal Recovery Threshold of Coded Matrix Multiplication
Dutta, S., Fahim, M., Haddadpour, F., Jeong, H., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2929328 · Cited by 285 · see citing works
Coded Computation Over Heterogeneous Clusters
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006961 · Cited by 269 · see citing works
Self-citationDevice Scheduling with Fast Convergence for Wireless Federated Learning
Shi, W., Zhou, S., & Niu, Z. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9149138 · Cited by 216 · see citing works
Motivating Workers in Federated Learning: A Stackelberg Game Perspective
Sarikaya, Y., & Erçetin, Ö. (2019) · IEEE Networking Letters · doi:10.1109/lnet.2019.2947144 · Cited by 209 · see citing works
Hu, S., Chen, X., Ni, W., Hossain, E., & Wang, X. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3086014 · Cited by 178 · see citing works
A Unified Coding Framework for Distributed Computing with Straggling Servers
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2016.7848828 · Cited by 176 · see citing works
Efficient Parallel Split Learning Over Resource-Constrained Wireless Edge Networks
Lin, Z., Zhu, G., Deng, Y., Chen, X., Gao, Y., Huang, K., & Fang, Y. M. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3359040 · Cited by 169 · see citing works
Handling privacy-sensitive medical data with federated learning: Challenges and future directions
Aouedi, O., Sacco, A., Piamrat, K., & Marchetto, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3185673 · Cited by 165 · see citing works
Gradient Coding From Cyclic MDS Codes and Expander Graphs
Raviv, N., Tamo, I., Tandon, R., & Dimakis, A. G. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3029396 · Cited by 158 · see citing works
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously
Li, S., Yu, M., Yang, C.-S., Avestimehr, A. S., Kannan, S., & Viswanath, P. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174305 · Cited by 160 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
Provably Secure Federated Learning against Malicious Clients
Cao, X., Jia, J., & Gong, N. Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16849 · Cited by 141 · see citing works
An Exact Quantized Decentralized Gradient Descent Algorithm
Reisizadeh, A., Mokhtari, A., Hassani, H., & Pedarsani, R. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2932876 · Cited by 138 · see citing works
Self-citationReisizadeh, A., Tziotis, I., Hassani, H., Mokhtari, A., & Pedarsani, R. (2022) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2022.3205475 · Cited by 137 · see citing works
Self-citationA Scalable Framework for Wireless Distributed Computing
Li, S., Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2017.2702605 · Cited by 130 · see citing works
Ng, J. S., Lim, W. Y. B., Luong, N. C., Xiong, Z., Asheralieva, A., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3091684 · Cited by 143 · see citing works
Speeding up distributed machine learning using codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2016) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541478 · Cited by 125 · see citing works
Self-citationReliable Distributed Computing for Metaverse: A Hierarchical Game-Theoretic Approach
Jiang, Y., Kang, J., Niyato, D. T., Ge, X., Xiong, Z., Miao, C., & Shen, X. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3204839 · Cited by 124 · see citing works
A Survey on Resource Management in Joint Communication and Computing-Embedded SAGIN
Chen, Q., Guo, Z., Meng, W., Han, S., Li, C., & Quek, T. Q. S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3421523 · Cited by 129 · see citing works
Coded Sparse Matrix MultiplicationPreprint
Wang, S., Liu, J., & Shroff, N. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03430 · Cited by 116 · see citing works
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication
Mallick, A., Chaudhari, M., Sheth, U., Palanikumar, G., & Joshi, G. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366706 · Cited by 114 · see citing works
Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding
Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437563 · Cited by 111 · see citing works
Towards 6G hyper-connectivity: Vision, challenges, and key enabling technologies
Lee, H., Lee, B., Yang, H., Kim, J., Kim, S., Shin, W., Shim, B., & Poor, H. V. (2023) · Journal of Communications and Networks · doi:10.23919/jcn.2023.000006 · Cited by 111 · see citing works
Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks
Prakash, S., Dhakal, S., Akdeniz, M. R., Yona, Y., Talwar, S., Avestimehr, S., & Himayat, N. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036961 · Cited by 109 · see citing works
The Capacity of Private Computation
Sun, H., & Jafar, S. A. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2888494 · Cited by 102 · see citing works
Near-Optimal Straggler Mitigation for Distributed Gradient Methods
Li, S.-Z., Kalan, S. M. M., Avestimehr, A., & Soltanolkotabi, M. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2018.00137 · Cited by 105 · see citing works
Block-Diagonal and LT Codes for Distributed Computing With Straggling Servers
Severinson, A., Amat, A. G. I., & Rosnes, E. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2877391 · Cited by 93 · see citing works
Private and Secure Distributed Matrix Multiplication With Flexible Communication Load
Aliasgari, M., Simeone, O., & Kliewer, J. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2972166 · Cited by 92 · see citing works
Numerically Stable Polynomially Coded Computing
Fahim, M., & Cadambe, V. R. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849468 · Cited by 91 · see citing works
Private Retrieval, Computing, and Learning: Recent Progress and Future Challenges
Ulukuş, Ş., Avestimehr, S., Gastpar, M., Jafar, S. A., Tandon, R., & Tian, C. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3142358 · Cited by 90 · see citing works
Yu, Q., & Avestimehr, A. S. (2020) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174167 · Cited by 84 · see citing works
Stochastic Gradient Coding for Straggler Mitigation in Distributed Learning
Bitar, R., Wootters, M. K., & Rouayheb, S. E. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991361 · Cited by 4 · see citing works
Hierarchical Coding for Distributed Computing
Park, H., Lee, K., Sohn, J.-Y., Suh, C., & Moon, J. J. (2018) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2018.8437669 · Cited by 82 · see citing works
Self-citationSpeeding Up Distributed Gradient Descent by Utilizing Non-persistent Stragglers
Özfatura, E., Gündüz, D., & Ulukuş, Ş. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849684 · Cited by 82 · see citing works
Sun, Y., Zhou, S., & Niu, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3030889 · Cited by 78 · see citing works
Dhakal, S., Prakash, S., Yona, Y., Talwar, S., & Himayat, N. (2019) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps45667.2019.9024521 · Cited by 77 · see citing works
Accelerating Split Federated Learning Over Wireless Communication Networks
Xu, C., Li, J., Liu, Y., Ling, Y., & Wen, M. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3327372 · Cited by 77 · see citing works
Li, K., Tao, M., & Chen, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2985039 · Cited by 76 · see citing works
Minimizing latency for secure distributed computing
Bitar, R., Parag, P., & Rouayheb, S. E. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8007060 · Cited by 72 · see citing works
Latency Minimization for Mobile Edge Computing Networks
Chen, C.-L., Brinton, C. G., & Aggarwal, V. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3117511 · Cited by 74 · see citing works
Adaptive Hierarchical Federated Learning Over Wireless Networks
Xu, B., Xia, W., Wen, W., Liu, P., Zhao, H., & Zhu, H. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3135541 · Cited by 74 · see citing works
Zhang, Y., Liu, D., Duan, M., Li, L., Chen, X., Ren, A., Tan, Y., & Wang, C. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3237752 · Cited by 73 · see citing works
Cross Subspace Alignment Codes for Coded Distributed Batch Computation
Jia, Z., & Jafar, S. A. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3064827 · Cited by 73 · see citing works
Communicate to Learn at the Edge
Gündüz, D., Kurka, D. B., Jankowski, M., Amiri, M. M., Özfatura, E., & Sreekumar, S. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000394 · Cited by 69 · see citing works
CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning
Schlegel, R., Kumar, S., Rosnes, E., & Amat, A. G. I. (2023) · IEEE Transactions on Communications · doi:10.1109/tcomm.2023.3244243 · Cited by 71 · see citing works
Jia, Z., & Jafar, S. A. (2020) · IEEE Transactions on Information Theory · doi:10.1109/tit.2020.3013152 · Cited by 71 · see citing works
Computation Scheduling for Distributed Machine Learning With Straggling Workers
Amiri, M. M., & Gündüz, D. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2952051 · Cited by 68 · see citing works
Secure Coded Multi-Party Computation for Massive Matrix Operations
Nodehi, H. A., & Maddah-Ali, M. A. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3050853 · Cited by 71 · see citing works
Autonomic machine learning platform
Lee, K. M., Yoo, J., Kim, S., Lee, J.-H., & Hong, J. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.07.003 · Cited by 66 · see citing works
Self-citationYu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2017.8262778 · Cited by 65 · see citing works
On the Optimality of D2D Coded Caching With Uncoded Cache Placement and One-Shot Delivery
Yapar, Ç., Wan, K., Schaefer, R. F., & Caire, G. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2940671 · Cited by 63 · see citing works
Wireless MapReduce Distributed Computing
Li, F., Chen, J., & Wang, Z. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2924621 · Cited by 65 · see citing works
Distributed Optimization of Collaborative Regions in Large-Scale Inhomogeneous Fog Computing
Lyu, X., Ren, C., Ni, W., Tian, H., & Liu, R. P. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2815359 · Cited by 62 · see citing works
Timely Distributed Computation With Stragglers
Buyukates, B., & Ulukuş, Ş. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3001873 · Cited by 61 · see citing works
Dhakal, S., Saurav, P., Yona, Y., Talwar, S., & Nageen, H. (2019) · IEEE Conference Proceedings · Cited by 60 · see citing works
Analog Lagrange Coded Computing
Soleymani, M., Mahdavifar, H., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3056377 · Cited by 58 · see citing works
Minimizing Latency for Secure Coded Computing Using Secret Sharing via Staircase Codes
Bitar, R., Parag, P., & Rouayheb, S. E. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2988506 · Cited by 57 · see citing works
CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework
Zhang, Y., Duan, M., Liu, D., Li, L., Ren, A., Chen, X., Tan, Y., & Wang, C. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533794 · Cited by 55 · see citing works
OverSketch: Approximate Matrix Multiplication for the Cloud
Gupta, V., Wang, S., Courtade, T. A., & Ramchandran, K. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622139 · Cited by 55 · see citing works
Self-citationGuo, H., Wang, H., Song, T., Yang, H., Lv, Z., Jin, X., Xue, Z., Ma, R., & Guan, H. (2021) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3472883.3486990 · Cited by 54 · see citing works
Anytime coding for distributed computation
Ferdinand, N. S., & Draper, S. C. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852337 · Cited by 54 · see citing works
Universally Decodable Matrices for Distributed Matrix-Vector Multiplication
Ramamoorthy, A., Tang, L., & Vontobel, P. O. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849451 · Cited by 54 · see citing works
A Hierarchical Incentive Design Toward Motivating Participation in Coded Federated Learning
Ng, J. S., Lim, W. Y. B., Xiong, Z., Cao, X., Niyato, D. T., Leung, C., & Kim, D. I. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126057 · Cited by 54 · see citing works
Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse GradientsPreprint
Mitra, A., Jaafar, R. H., Pappas, G. J., & Hassani, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07053 · Cited by 53 · see citing works
Li, S., & Avestimehr, S. (2020) · Foundations and Trends® in Communications and Information Theory · doi:10.1561/0100000103 · Cited by 56 · see citing works
Computation Scheduling for Distributed Machine Learning with Straggling Workers
Amiri, M. M., & Gündüz, D. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682911 · Cited by 50 · see citing works
CodedSketch: A Coding Scheme for Distributed Computation of Approximated Matrix Multiplication
Jahani-Nezhad, T., & Maddah-Ali, M. A. (2021) · IEEE Transactions on Information Theory · doi:10.1109/tit.2021.3068165 · Cited by 50 · see citing works
Polynomially Coded Regression: Optimal Straggler Mitigation via Data EncodingPreprint
Li, S., Kalan, S. M. M., Yu, Q., Soltanolkotabi, M., & Avestimehr, A. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09934 · Cited by 50 · see citing works
Robust Gradient Descent via Moment Encoding and LDPC Codes
Maity, R. K., Rawat, A. S., & Mazumdar, A. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849514 · Cited by 49 · see citing works
C3LES: Codes for Coded Computation that Leverage Stragglers
Das, A. B., Tang, L., & Ramamoorthy, A. (2018) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2018.8613321 · Cited by 49 · see citing works
Communication-Aware Scheduling of Serial Tasks for Dispersed Computing
Yang, C.-S., Pedarsani, R., & Avestimehr, A. S. (2019) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2019.2919553 · Cited by 49 · see citing works
Self-citationSecure Distributed Matrix Computation With Discrete Fourier Transform
Mital, N., Ling, C., & Gündüz, D. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3158868 · Cited by 49 · see citing works
A Comprehensive Survey on Training Acceleration for Large Machine Learning Models in IoT
Wang, H., Qu, Z., Zhou, Q., Zhang, H., Luo, B., Xu, W., Guo, S., & Li, R. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3111624 · Cited by 49 · see citing works
Parity models: erasure-coded resilience for prediction serving systems
Kosaian, J., Rashmi, K. V., & Venkataraman, S. (2019) · Proceedings of the 27th ACM Symposium on Operating Systems Principles · doi:10.1145/3341301.3359654 · Cited by 48 · see citing works
Coded Federated Computing in Wireless Networks with Straggling Devices and Imperfect CSI
Ha, S., Zhang, J., Simeone, O., & Kang, J. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849507 · Cited by 48 · see citing works
GCSA Codes With Noise Alignment for Secure Coded Multi-Party Batch Matrix Multiplication
Chen, Z., Jia, Z., Wang, Z., & Jafar, S. A. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3052934 · Cited by 47 · see citing works
Managing Distributed Machine Learning Lifecycle for Healthcare Data in the Cloud
Zeydan, E., Arslan, Ş. S., & Liyanage, M. (2024) · IEEE Access · doi:10.1109/access.2024.3443520 · Cited by 47 · see citing works
Random Khatri-Rao-Product Codes for Numerically-Stable Distributed Matrix Multiplication
Subramaniam, A. M., Heidarzadeh, A., & Narayanan, K. R. (2019) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2019.8919859 · Cited by 47 · see citing works
Distributed Learning for Wireless Communications: Methods, Applications and Challenges
Qian, L., Yang, P., Xiao, M., Dobre, O. A., Renzo, M. D., Li, J., Han, Z., Qin, Y., & Zhao, J. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3156756 · Cited by 46 · see citing works
Coded computation for multicore setups
Lee, K., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006962 · Cited by 45 · see citing works
Self-citationStraggler-Aware Distributed Learning: Communication–Computation Latency Trade-Off
Özfatura, E., Ulukuş, Ş., & Gündüz, D. (2020) · Entropy · doi:10.3390/e22050544 · Cited by 45 · see citing works
Aktaş, M. F., & Soljanin, E. (2019) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2019.2946464 · Cited by 44 · see citing works
Ali, S., Wang, J., & Leung, V. C. M. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102922 · Cited by 55 · see citing works
Coded Distributed Computing: Straggling Servers and Multistage Dataflows
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852225 · Cited by 43 · see citing works
Distributed Gradient Descent with Coded Partial Gradient Computations
Özfatura, E., Ulukuş, Ş., & Gündüz, D. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683267 · Cited by 43 · see citing works
Fundamental Limits of Approximate Gradient Coding
Wang, S., Liu, J., & Shroff, N. B. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3366700 · Cited by 43 · see citing works
Storage, Computation, and Communication: A Fundamental Tradeoff in Distributed Computing
Yan, Q., Yang, S., & Wigger, M. (2018) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2018.8613519 · Cited by 43 · see citing works
Boomerang: Redundancy Improves Latency and Throughput in Payment-Channel Networks
Bagaria, V., Neu, J., & Tse, D. N. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-51280-4_17 · Cited by 43 · see citing works
A Double Auction Mechanism for Resource Allocation in Coded Vehicular Edge Computing
Ng, J. S., Lim, W. Y. B., Xiong, Z., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3131395 · Cited by 43 · see citing works
Gradient Coding with Clustering and Multi-Message Communication
Özfatura, E., Gündüz, D., & Ulukuş, Ş. (2019) · IEEE Data Science Workshop (DSW) · doi:10.1109/dsw.2019.8755563 · Cited by 42 · see citing works
FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction
Damaskinos, G., Guerraoui, R., Kermarrec, A.-M., Nitu, V., Patra, R., & Taı̈ani, F. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3527621 · Cited by 42 · see citing works
- Authors
- Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- University of California, Berkeley (US)
- University of California, Santa Barbara (US)
- University of Wisconsin–Madison (US)
- Venue
- IEEE Transactions on Information Theory
- Published
- 2018-03
- Volume
- 64
- Issue
- 3
- Pages
- 1514-1529
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2268702383
- Cited by
- 829
- References
- 122
- Citations by year
- 2026: 24 · 2025: 56 · 2024: 79 · 2023: 101 · 2022: 111 · 2021: 148 · 2020: 130 · 2019: 105 · 2018: 48 · 2017: 16 · 2016: 11
- FWCI
- 54.46
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Information Theory (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute for Information & communications Technology Promotion(IITP) grant
- Korea government(MSIT) (2017-0-00694)
- Brain Korea 21 Plus Project, and NSF CIF grant (1703678)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- distributed learning, coded computing, straggler nodes, distributed matrix multiplication, data shuffling, communication bottleneck, multicast, caching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
