The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Speeding Up Distributed Machine Learning Using Codes
Open accessAbstract
Codes are widely used in many engineering applications to offerrobustnessagainstnoise. In large-scale systems, there are several types of noise that can affect the performance of distributed machine learning algorithms—straggler nodes, system failures, or communication bottlenecks—but there has been little interaction cutting across codes, machine learning, and distributed systems. In this paper, we provide theoretical insights on howcodedsolutions can achieve significant gains compared with uncoded ones. We focus on two of the most basic building blocks of distributed learning algorithms:matrix multiplicationanddata shuffling. For matrix multiplication, we use codes to alleviate the effect of stragglers and show that if the number of homogeneous workers is$n$, and the runtime of each subtask has an exponential tail, coded computation can speed up distributed matrix multiplication by a factor of$\log n$. For data shuffling, we use codes to reduce communication bottlenecks, exploiting the excess in storage. We show that when a constant fraction$\alpha $of the data matrix can be cached at each worker, and$n$is the number of workers,coded shufflingreduces the communication cost by a factor of$\left({\alpha + \frac {1}{n}}\right)\gamma (n)$compared with uncoded shuffling, where$\gamma (n)$is the ratio of the cost of unicasting$n$messages to$n$users to multicasting a common message (of the same size) to$n$users. For instance,$\gamma (n) \simeq n$if multicasting a message to$n$users is as cheap as unicasting a message to one user. We also provide experimental results, corroborating our theoretical gains of the coded algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Data poisoning attacks may occur in two ways such as clean-label attack (adversary can poison correct class of data sample) and dirty-label attack (adversary try to misclassify the target label of training dataset) [147], [148].”
From Challenges, Applications and Design Aspects of Federated Learning: A Survey (2021) · cites this paper as [148]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,917
Smith, D. K., & Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 12,385
(2010) · Cited by 4,411
Distributed Subgradient Methods for Multi-Agent Optimization
Nedic, A., & Ozdaglar, A. (2009) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2008.2009515 · Cited by 3,719
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
Weik, M. H. (2000) · Springer eBooks · doi:10.1007/1-4020-0613-6_12436 · Cited by 2,700
Elements of Information Theory
Nalewajski, R. F. (2011) · Springer eBooks · doi:10.1007/978-3-642-20180-6_8 · Cited by 2,586
Stochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_25 · Cited by 2,168
Network Coding for Distributed Storage Systems
Dimakis, A. G., Godfrey, P. B., Wu, Y., Wainwright, M. J., & Ramchandran, K. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2054295 · Cited by 1,913
Self-citationMLlib: Machine Learning in Apache SparkPreprint
Meng, X., Bradley, J. K., Yavuz, B., Sparks, E., Venkataraman, S., Liu, D., Freeman, J., Tsai, D., Amde, M., Owen, S., et al. (2015) · Journal of machine learning research · doi:10.48550/arxiv.1505.06807 · Cited by 1,898
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852
Low, Y., Bickson, D., Gonzalez, J. E., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2212351.2212354 · Cited by 1,755
Maddah-Ali, M. A., & Niesen, U. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2306938 · Cited by 1,726
Improving MapReduce performance in heterogeneous environments
Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., & Stoica, I. G. (2008) · doi:10.5555/1855741.1855744 · Cited by 1,625
Costello, D. J., & Shu, L. (2004) · Cited by 1,564
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211
Scaling distributed machine learning with the parameter server
Li, M., Andersen, D. G., Park, J. W., Smola, A. J., Ahmed, A. A., Josifovski, V., Long, J. E., Shekita, E. J., & Su, B.-Y. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685095 · Cited by 1,182
Parallelized Stochastic Gradient Descent
Zinkevich, M., Weimer, M., Li, L., & Smola, A. J. (2010) · Cited by 1,124
Scaling Distributed Machine Learning with the Parameter Server
Li, M. (2014) · International Conference on Big Data Science and Computing (BigDataScience) · doi:10.1145/2640087.2644155 · Cited by 1,010
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2012) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2011.2161027 · Cited by 947
Erasure coding in windows azure storage
Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., & Yekhanin, S. (2012) · Cited by 928
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
Duchi, J. C., Agarwal, A., & Wainwright, M. J. (2010) · Cited by 884
On the Locality of Codeword Symbols
Gopalan, P., Huang, C., Simitci, H., & Yekhanin, S. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2012.2208937 · Cited by 836
Rashmi, K. V., Shah, N. B., & Kumar, P. V. (2011) · NOT FOUND REPOSITORY (Indian Institute of Science Bangalore) · Cited by 701
Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., Dimakis, A. G., Vadali, R., Chen, S., & Borthakur, D. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2535573.2488339 · Cited by 690
Self-citationDiffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks
Chen, J., & Sayed, A. H. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2012.2198470 · Cited by 684
Decentralized Coded Caching Attains Order-Optimal Memory-Rate Tradeoff
Maddah-Ali, M. A., & Niesen, U. (2014) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2014.2317316 · Cited by 667
Reining in the outliers in map-reduce clusters using Mantri
Ananthanarayanan, G., Kandula, S., Greenberg, A. G., Stoica, I. G., Lu, Y., Saha, B., & Harris, E. F. (2010) · Operating Systems Design and Implementation · doi:10.5555/1924943.1924962 · Cited by 660
Rashmi, K. V., Shah, N. B., & Kumar, P. V. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2159049 · Cited by 589
Index Coding With Side Information
Bar-Yossef, Z., Birk, Y., Jayram, T. S., & Kol, T. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2103753 · Cited by 474
Communication-optimal Parallel and Sequential QR and LU Factorizations
Demmel, J. W., Grigori, L., Hoemmen, M. F., & Langou, J. (2012) · SIAM Journal on Scientific Computing · doi:10.1137/080731992 · Cited by 428
Effective straggler mitigation: attack of the clones
Ananthanarayanan, G., Ghodsi, A., Shenker, S. J., & Stoica, I. G. (2013) · Cited by 422
Parallel stochastic gradient algorithms for large-scale matrix completion
Recht, B., & Ré, C. (2013) · Mathematical Programming Computation · doi:10.1007/s12532-013-0053-8 · Cited by 378
“Short-Dot”: Computing Large Linear Transforms Distributedly Using Coded Short Dot Products
Dutta, S., Cadambe, V. R., & Grover, P. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2927558 · Cited by 361
Birk, Y., & Kol, T. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.874540 · Cited by 338
Distributed delayed stochastic optimization
Agarwal, A., & Duchi, J. C. (2012) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426626 · Cited by 318
Augmenting data center networks with multi-gigabit wireless links
Halperin, D., Kandula, S., Padhye, J. D., Bahl, P., & Wetherall, D. J. (2011) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2018436.2018442 · Cited by 312
MLbase: A Distributed Machine-learning System
Kraska, T., Talwalkar, A., & Duchi, J. C. (2013) · Cited by 307
Optimal linear codes with a local-error-correction property
Prakash, N., Kamath, G. M., Lalitha, V., & Kumar, P. V. (2012) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2012.6284028 · Cited by 304
Huang, C., Chen, M., & Li, J. (2007) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2007.37 · Cited by 300
Bekkerman, R., Bilenko, M., & Langford, J. C. (2011) · doi:10.1145/2107736.2107740 · Cited by 290
Coded Computation Over Heterogeneous Clusters
Reisizadeh, A., Prakash, S., Pedarsani, R., & Avestimehr, A. S. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006961 · Cited by 269
Self-citationRashmi, K. V., Shah, N. B., Gu, D., Kuang, H., Borthakur, D., & Ramchandran, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.0186 · Cited by 266
Self-citationSelf-repairing homomorphic codes for distributed storage systems
Oggier, F., & Datta, A. (2011) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2011.5934901 · Cited by 261
High-dimensional coded matrix multiplication
Lee, K., Suh, C., & Ramchandran, K. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006963 · Cited by 248
Self-citation1Optimal Locally Repairable and Secure Codes for Distributed Storage SystemsPreprint
Rawat, A. S., Koyluoglu, O. O., Silberstein, N., & Vishwanath, S. (2016) · Cited by 224
Papailiopoulos, D., & Dimakis, A. G. (2012) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2012.6284027 · Cited by 212
Self-citationA "hitchhiker's" guide to fast and efficient data reconstruction in erasure-coded data centers
Rashmi, K. V., Shah, N. B., Gu, D., Kuang, H., Borthakur, D., & Ramchandran, K. (2014) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2619239.2626325 · Cited by 209
Self-citationOptimal Locally Repairable and Secure Codes for Distributed Storage Systems
Rawat, A. S., Koyluoglu, O. O., Silberstein, N., & Vishwanath, S. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2013.2288784 · Cited by 200
Exact-Repair MDS Code Construction Using Interference Alignment
Suh, C., & Ramchandran, K. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2105003 · Cited by 200
Self-citationSimple regenerating codes: Network coding for cloud storage
Papailiopoulos, D., Luo, J., Dimakis, A. G., Huang, C., & Li, J. (2012) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2012.6195703 · Cited by 197
Self-citationBekkerman, R., Bilenko, M., & Langford, J. C. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9781139042918 · Cited by 193
On the Delay-Storage Trade-Off in Content Download from Coded Distributed Storage Systems
Joshi, G., Liu, Y., & Soljanin, E. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.140518 · Cited by 191
A Unified Coding Framework for Distributed Computing with Straggling Servers
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2016.7848828 · Cited by 176
MLI: An API for Distributed Machine Learning
Sparks, E., Talwalkar, A., Smith, V., Kottalam, J., Pan, X., Gonzalez, J. E., Franklin, M. J. M., Jordan, M. I., & Kraska, T. (2013) · Proceedings · doi:10.1109/icdm.2013.158 · Cited by 154
Augmenting data center networks with multi-gigabit wireless links
Halperin, D., Kandula, S., Padhye, J. D., Bahl, P., & Wetherall, D. J. (2011) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2043164.2018442 · Cited by 137
Zhang, C., & Ré, C. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732977.2733001 · Cited by 133
Reducing Latency via Redundant Requests
Gardner, K., Zbarsky, S., Doroudi, S., Harchol‐Balter, M., & Esa Hyytiä, A. (2015) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/2745844.2745873 · Cited by 131
Efficient Redundancy Techniques for Latency Reduction in Cloud Systems
Joshi, G., Soljanin, E., & Wornell, G. W. (2017) · ACM Transactions on Modeling and Performance Evaluation of Computing Systems · doi:10.1145/3055281 · Cited by 130
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447112 · Cited by 128
Speeding up distributed machine learning using codes
Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2016) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541478 · Cited by 125
Self-citationDistributed Delayed Stochastic OptimizationPreprint
Agarwal, A., & Duchi, J. C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1104.5525 · Cited by 117
Why random reshuffling beats stochastic gradient descent
Gürbüzbalaban, M., Ozdaglar, A., & Parrilo, P. A. (2019) · Mathematical Programming · doi:10.1007/s10107-019-01440-w · Cited by 110
XORing Elephants: Novel Erasure Codes for Big DataPreprint
Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., Dimakis, A. G., Vadali, R., Chen, S., & Borthakur, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3791 · Cited by 102
Self-citationCodes can reduce queueing delay in data centers
Huang, L., Pawar, S., Zhang, H., & Ramchandran, K. (2012) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2012.6284026 · Cited by 99
Self-citationA "hitchhiker's" guide to fast and efficient data reconstruction in erasure-coded data centers
Rashmi, K. V., Shah, N. B., Gu, D., Kuang, H., Borthakur, D., & Ramchandran, K. (2014) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2740070.2626325 · Cited by 87
Self-citationEfficient task replication for fast response times in parallel computation
Wang, D., Joshi, G., & Wornell, G. W. (2014) · doi:10.1145/2591971.2592042 · Cited by 86
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2015) · Cited by 85
Repair optimal erasure codes through hadamard designs
Papailiopoulos, D., Dimakis, A. G., & Cadambe, V. R. (2011) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2011.6120328 · Cited by 84
Self-citationFundamental limits of distributed caching in D2D wireless networks
Ji, M., Caire, G., & Molisch, A. F. (2013) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw.2013.6691247 · Cited by 82
Reducing Latency via Redundant Requests
Gardner, K., Zbarsky, S., Doroudi, S., Harchol‐Balter, M., & Esa Hyytiä, A. (2015) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2796314.2745873 · Cited by 82
Karamchandani, N., Niesen, U., Maddah-Ali, M. A., & Diggavi, S. N. (2014) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2014.6875212 · Cited by 78
BLAS (Basic Linear Algebra Subprograms).
Geijn, R. A., & Goto, K. (2011) · Cited by 77
Zhu, Y., Zhou, X., Zhang, Z., Zhou, L., Vahdat, A. M., Zhao, B. Y., & Zheng, H.-T. (2014) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/2639108.2639140 · Cited by 75
TOFEC: Achieving optimal throughput-delay trade-off of cloud storage using erasure codes
Liang, G., & Kozat, U. C. (2014) · doi:10.1109/infocom.2014.6848010 · Cited by 75
Reliable Memories with Subline Accesses
Han, J., & Lastras-Montaño, L. A. (2007) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2007.4557599 · Cited by 73
The MDS queue: Analysing the latency performance of erasure codes
Shah, N. B., Lee, K., & Ramchandran, K. (2014) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2014.6874955 · Cited by 72
Self-citationMinimizing latency for secure distributed computing
Bitar, R., Parag, P., & Rouayheb, S. E. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8007060 · Cited by 72
When do redundant requests reduce latency ?
Shah, N. B., Lee, K., & Ramchandran, K. (2013) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2013.6736597 · Cited by 70
Self-citationPedarsani, R., Maddah-Ali, M. A., & Niesen, U. (2014) · International Conference on Communications · doi:10.1109/icc.2014.6883597 · Cited by 68
Self-citationError resilience in distributed storage via rank-metric codes
Silberstein, N., Rawat, A. S., & Vishwanath, S. (2012) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2012.6483348 · Cited by 67
Arslan, M. Y., Singh, I., Singh, S. N., Madhyastha, H. V., Sundaresan, K., & Krishnamurthy, S. V. (2012) · ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) · doi:10.1145/2413176.2413199 · Cited by 66
MDS array codes with optimal rebuilding
Tamo, I., Wang, Z., & Bruck, J. (2011) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2011.6033733 · Cited by 65
Kamath, G. M., Prakash, N., Lalitha, V., & Kumar, P. V. (2012) · NOT FOUND REPOSITORY (Indian Institute of Science Bangalore) · doi:10.48550/arxiv.1211.1932 · Cited by 64
Fundamental tradeoff between computation and communication in distributed computing
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541612 · Cited by 63
Anytime coding for distributed computation
Ferdinand, N. S., & Draper, S. C. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852337 · Cited by 54
Optimal Repair of MDS Codes in Distributed Storage via Subspace Interference AlignmentPreprint
Cadambe, V. R., Huang, C., Jafar, S. A., & Li, J. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1106.1250 · Cited by 50
Blake, I. F. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009283403.008 · Cited by 49
The MDS Queue: Analysing the Latency Performance of Erasure Codes
Lee, K., Shah, N. B., Huang, L., & Ramchandran, K. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2674671 · Cited by 49
Self-citationCoded computation for multicore setups
Lee, K., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006962 · Cited by 45
Self-citationKamath, G. M., Prakash, N., Lalitha, V., & Kumar, P. V. (2013) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2013.6620498 · Cited by 44
Gradient CodingPreprint
Tandon, R., Lei, Q., Dimakis, A. G., & Karampatziakis, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03301 · Cited by 41
Communication-avoiding parallel strassen: implementation and performance
Lipshitz, B., Ballard, G., Demmel, J. W., & Schwartz, O. (2012) · doi:10.5555/2388996.2389133 · Cited by 39
Efficient task replication for fast response times in parallel computation
Wang, D., Joshi, G., & Wornell, G. W. (2014) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2637364.2592042 · Cited by 37
Communication-avoiding algorithms for linear algebra and beyond
Demmel, J. W. (2013) · Proceedings - IEEE International Parallel and Distributed Processing Symposium · doi:10.1109/ipdps.2013.123 · Cited by 36
Analyzing the download time of availability codes
Kadhe, S., Soljanin, E., & Sprintson, A. (2015) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2015.7282699 · Cited by 35
Information Theoretic Limits of Data Shuffling for Distributed Learning
Attia, M. A., & Tandon, R. (2016) · Globecom · doi:10.1109/glocom.2016.7841903 · Cited by 34
MLI: An API for Distributed Machine LearningPreprint
Sparks, E., Talwalkar, A., Smith, V., Kottalam, J., Pan, X., Gonzalez, J. E., Franklin, M. J. M., Jordan, M. I., & Kraska, T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.5426 · Cited by 34
Provably delay efficient data retrieving in storage clouds
Sun, Y. J., Zheng, Z., Koksal, C. E., Kim, K.-H., & Shroff, N. B. (2015) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2015.7218426 · Cited by 32
MDS Array Codes with Optimal RebuildingPreprint
Tamo, I., Wang, Z., & Bruck, J. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.3737 · Cited by 30
Fundamental Limits of Distributed Caching in D2D Wireless NetworksPreprint
Ji, M., Caire, G., & Molisch, A. F. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5856 · Cited by 27
A Fundamental Tradeoff between Computation and Communication in Distributed ComputingPreprint
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.07086 · Cited by 25
A pliable index coding approach to data shuffling
Song, L., Fragouli, C., & Zhao, T. (2017) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2017.8006991 · Cited by 25
Hierarchical Coded CachingPreprint
Karamchandani, N., Niesen, U., Maddah-Ali, M. A., & Diggavi, S. N. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.7007 · Cited by 23
Codes Can Reduce Queueing Delay in Data CentersPreprint
Huang, L., Pawar, S., Zhang, H., et al. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1202.1359 · Cited by 18
On the worst-case communication overhead for distributed data shuffling
Attia, M. A., & Tandon, R. (2016) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2016.7852338 · Cited by 16
A Unified Coding Framework for Distributed Computing with Straggling ServersPreprint
Li, S., Maddah-Ali, M. A., & Avestimehr, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.01690 · Cited by 16
On index coding with side information
Dau, S. H. (2012) · doi:10.32657/10356/48642 · Cited by 15
When do the availability codes make the stored data more available?
Kadhe, S., Soljanin, E., & Sprintson, A. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447111 · Cited by 15
DimmWitted: A Study of Main-Memory Statistical AnalyticsPreprint
Zhang, C., & Ré, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.7550 · Cited by 15
Author retrospective for optimizing matrix multiply using PHiPAC
Bilmes, J., Asanović, K., Chin, C., & Demmel, J. (2014) · International Conference on Supercomputing · doi:10.1145/2591635.2591656 · Cited by 13
Replicated Data Placement for Uncertain Scheduling
Chaubey, M., & Saule, É. (2015) · doi:10.1109/ipdpsw.2015.50 · Cited by 12
On scheduling redundant requests with cancellation overheads
Lee, K., Pedarsani, R., & Ramchandran, K. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7446991 · Cited by 12
Self-citationComputing while Charging: Building a Distributed Computing Infrastructure using Smartphones
Bakiadarshani, J. (2017) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2017.3060 · Cited by 6
A Pliable Index Coding Approach to Data ShufflingPreprint
Song, L., Fragouli, C., & Zhao, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.05540 · Cited by 1
Flagged first, then by citations
Research works printed but not linked: 3
- 10.1109/ISIT.2016.7541478 · DOI
- 2015, StarCluster
- 2015, Open MPI Open Source High Performance Computing
Cited by
Sorted by citations
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587
Self-citationMachine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
A Fundamental Tradeoff Between Computation and Communication in Distributed Computing
Li, S., Maddah-Ali, M. A., Yu, Q., & Avestimehr, A. S. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2756959 · Cited by 458
Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix MultiplicationPreprint
Yu, Q., Maddah-Ali, M. A., & Avestimehr, A. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10464 · Cited by 439
Metadata
- Authors
- Lee, K., Lam, M., Pedarsani, R., Papailiopoulos, D., & Ramchandran, K.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- University of California, Berkeley (US)
- University of California, Santa Barbara (US)
- University of Wisconsin–Madison (US)
- Venue
- IEEE Transactions on Information Theory
- Published
- 2018-03
- Volume
- 64
- Issue
- 3
- Pages
- 1514-1529
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2268702383
- Cited by
- 829
- References
- 122
- Citations by year
- 2026: 24 · 2025: 56 · 2024: 79 · 2023: 101 · 2022: 111 · 2021: 148 · 2020: 130 · 2019: 105 · 2018: 48
- FWCI
- 54.46
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Information Theory (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Institute for Information & communications Technology Promotion(IITP) grant
- Korea government(MSIT) (2017-0-00694)
- Brain Korea 21 Plus Project, and NSF CIF grant (1703678)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- distributed learning, coded computing, straggler nodes, distributed matrix multiplication, data shuffling, communication bottleneck, multicast, caching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
