The DOI matches its Crossref record; 125 of the 125 references it cites were found in a registry or an index.
A Survey on Distributed Machine Learning
Open accessAbstract
The demand for artificial intelligence has grown significantly over the past decade, and this growth has been fueled by advances in machine learning techniques and the ability to leverage hardware acceleration. However, to increase the quality of predictions and render machine learning solutions feasible for more complex applications, a substantial amount of training data is required. Although small machine learning models can be trained with modest amounts of data, the input for training larger models such as neural networks grows exponentially with the number of parameters. Since the demand for processing training data has outpaced the increase in computation power of computing machinery, there is a need for distributing the machine learning workload across multiple machines, and turning the centralized into a distributed system. These distributed systems present new challenges: first and foremost, the efficient parallelization of the training process and the creation of a coherent model. This article provides an extensive overview of the current state-of-the-art in the field by outlining the challenges and opportunities of distributed machine learning over conventional (centralized) machine learning, discussing the techniques used for distributed machine learning, and providing an overview of the systems that are available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,673
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
On the limited memory BFGS method for large scale optimization
Liu, D. C., & Nocedal, J. (1989) · Mathematical Programming · doi:10.1007/bf01589116 · Cited by 8,560
(2010) · Cited by 8,261
Methods of conjugate gradients for solving linear systems
Hestenes, M. R., & Stiefel, E. (1952) · Journal of Research of the National Bureau of Standards · doi:10.6028/jres.049.044 · Cited by 8,132
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003) · ACM SIGOPS Operating Systems Review · doi:10.1145/1165389.945450 · Cited by 4,984
The Hadoop Distributed File System
Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010) · 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) · doi:10.1109/msst.2010.5496972 · Cited by 4,834
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
(2010) · Cited by 4,411
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · Cited by 3,622
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 1×
“Unfortunately, eliminating both the bias and the variance is typically impossible, a phenomenon known as the bias-variance trade-o � [54].”
BackgroundEnd to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Popular Ensemble Methods: An Empirical Study
Opitz, D., & Maclin, R. (1999) · Journal of Artificial Intelligence Research · doi:10.1613/jair.614 · Cited by 2,953
Using MPI: Portable parallel programming with the message-passing interface
(1995) · Computers & Mathematics with Applications · doi:10.1016/0898-1221(95)90199-x · Cited by 2,854
Apache Spark: A unified engine for big data processing
Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M. J., et al. (2016) · Communications of the ACM · doi:10.1145/2934664 · Cited by 2,757
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · cited 1×
“[68] managed to train ResNet-50 in the span of 1 hour [61] using this approach, achieving linear scaling with the number of GPUs.”
MethodDeep Speech 2: End-to-End Speech Recognition in English and Mandarin
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,110
Probabilistic Latent Semantic Analysis
Hofmann, T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6705 · Cited by 2,047
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
A survey of rollback-recovery protocols in message-passing systems
Elnozahy, E. N. (., Alvisi, L., Wang, Y.-M., & Johnson, D. B. (2002) · ACM Computing Surveys · doi:10.1145/568522.568525 · Cited by 1,804
Graph-based approach for airborne light detection and ranging segmentation
Vilariño, D. L., Cabaleiro, J. C., Martínez, J., Rivera, F. F., & Pena, T. F. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.015020 · Cited by 1,721
Some Computer Organizations and Their Effectiveness
Flynn, M. J. (1972) · IEEE Transactions on Computers · doi:10.1109/tc.1972.5009071 · Cited by 1,713
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
Basic Linear Algebra Subprograms for Fortran Usage
Lawson, C. L., Hanson, R. J., Kincaid, D. R., & Krogh, F. T. (1979) · ACM Transactions on Mathematical Software · doi:10.1145/355841.355847 · Cited by 1,634
Distributed Systems: Concepts and Design
Coulouris, G., & Dollimore, J. (1988) · Cited by 1,629
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
Wright, S. J. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0892-3 · Cited by 1,438
Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003) · Proceedings of the nineteenth ACM symposium on Operating systems principles · doi:10.1145/945445.945450 · Cited by 1,423
Classifier Fitness Based on Accuracy
Wilson, S. W. (1995) · Evolutionary Computation · doi:10.1162/evco.1995.3.2.149 · Cited by 1,388
A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property
Dai, Y. H., & Yuan, Y. (1999) · SIAM Journal on Optimization · doi:10.1137/s1052623497318992 · Cited by 1,378
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Federated Learning of Deep Networks using Model AveragingPreprint
McMahan, H. B., Moore, E., Ramage, D., & y Arcas, B. A. (2016) · arXiv (Cornell University) · Cited by 1,288
Map-Reduce for Machine Learning on Multicore
Chu, C.-T., Kim, S. K., Lin, Y.-A., Yu, Y., Bradski, G., Ng, A. Y., & Olukotun, K. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0040 · Cited by 1,229
Theano: A CPU and GPU Math Compiler in Python
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., & Bengio, Y. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-003 · Cited by 1,052
Web search for a planet: the google cluster architecture
Barroso, L. A., Dean, J., & Holzle, U. (2003) · IEEE Micro · doi:10.1109/mm.2003.1196112 · Cited by 1,025
Scaling distributed machine learning with the parameter server
Li, M., Andersen, D. G., Park, J. W., Smola, A. J., Ahmed, A. A., Josifovski, V., Long, J. E., Shekita, E. J., & Su, B.-Y. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685095 · Cited by 1,021
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992
Scaling Distributed Machine Learning with the Parameter Server
Li, M. (2014) · Proceedings of the 2014 International Conference on Big Data Science and Computing · doi:10.1145/2640087.2644155 · Cited by 979
A survey of machine learning for big data processing
Qiu, J., Wu, Q., Ding, G., Xu, Y., & Feng, S. (2016) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-016-0355-x · Cited by 894
Consumer credit-risk models via machine-learning algorithms
Khandani, A. E., Kim, A. J., & Lo, A. W. (2010) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2010.06.001 · Cited by 864
Markov chain Monte Carlo method and its application
Brooks, S. (1998) · Journal of the Royal Statistical Society: Series D (The Statistician) · doi:10.1111/1467-9884.00117 · Cited by 843
An updated set of basic linear algebra subprograms (BLAS)
(2002) · ACM Transactions on Mathematical Software · doi:10.1145/567806.567807 · Cited by 834
Clearing the clouds: a study of emerging scale-out workloads on modern hardware
Ferdman, M., Adileh, A., Kocberber, O., Volos, S., Alisafaee, M., Jevdjic, D., Kaynak, C., Popescu, A. D., Ailamaki, A., & Falsafi, B. (2012) · Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/2150976.2150982 · Cited by 814
A survey of deep learning-based network anomaly detection
Kwon, D., Kim, H., Kim, J., Suh, S. C., Kim, I., & Kim, K. J. (2019) · Cluster Computing · doi:10.1007/s10586-017-1117-8 · Cited by 778
Traffic sign recognition with multi-scale Convolutional Networks
Sermanet, P., & LeCun, Y. (2011) · The 2011 International Joint Conference on Neural Networks · doi:10.1109/ijcnn.2011.6033589 · Cited by 759
Large-scale deep unsupervised learning using graphics processors
Raina, R., Madhavan, A., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553486 · Cited by 742 · cited 1×
“, [126], [18], [78]).”
(2013) · Cited by 651
Project Adam: building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., & Kalyanaraman, K. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685094 · Cited by 550
Revisiting Distributed Synchronous SGD
Pan, X., Chen, J., Monga, R., Bengio, S., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 548
High Performance Cluster Computing: Architectures and Systems
Buyya, R. (1999) · Prentice Hall PTR eBooks · Cited by 511
On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing
Foster, I., & Iamnitchi, A. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-45172-3_11 · Cited by 504
Understanding error propagation in deep learning neural network (DNN) accelerators and applications
Li, G., Hari, S. K. S., Sullivan, M., Tsai, T., Pattabiraman, K., Emer, J., & Keckler, S. W. (2017) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3126908.3126964 · Cited by 492
Google’s MapReduce programming model — Revisited
Lämmel, R. (2008) · Science of Computer Programming · doi:10.1016/j.scico.2007.07.001 · Cited by 489
Communication Efficient Distributed Machine Learning with the Parameter Server
Li, M., Andersen, D. G., Smola, A. J., & Yu, K. (2014) · neural information processing systems · Cited by 475
Horovod: fast and easy distributed deep learning in TensorFlow
Sergeev, A., & Balso, M. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05799 · Cited by 472
A Stochastic Quasi-Newton Method for Large-Scale Optimization
Byrd, R. H., Hansen, S. L., Nocedal, J., & Singer, Y. (2016) · SIAM Journal on Optimization · doi:10.1137/140954362 · Cited by 459
Seide, F., Fu, H., Droppo, J., Li, G., & Yu, D. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-274 · Cited by 443
A survey on platforms for big data analytics
Singh, D., & Reddy, C. K. (2015) · Journal of Big Data · doi:10.1186/s40537-014-0008-6 · Cited by 433
A survey of open source tools for machine learning with big data in the Hadoop ecosystem
Landset, S., Khoshgoftaar, T. M., Richter, A. N., & Hasanin, T. (2015) · Journal of Big Data · doi:10.1186/s40537-015-0032-1 · Cited by 432
LAPACK: A Portable Linear Algebra Library for High-Performance Computers
Anderson, E., Bai, Z., Bischof, C. H., Demmel, J. W., Dongarra, J. J., Croz, J. D., Greenbaum, A., Hammarling, S. J., McKenney, A., & Sorensen, D. C. (1990) · Cited by 428
GPU implementation of neural networks
Oh, K.-S., & Jung, K. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2004.01.013 · Cited by 420
Bandwidth optimal all-reduce algorithms for clusters of workstations
Patarasuk, P., & Yuan, X. (2009) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2008.09.002 · Cited by 409
Evaluation of machine learning classifiers for mobile malware detection
Narudin, F. A., Feizollah, A., Anuar, N. B., & Gani, A. (2016) · Soft Computing · doi:10.1007/s00500-014-1511-6 · Cited by 401
An architecture for parallel topic models
Smola, A., & Narayanamurthy, S. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920931 · Cited by 399
Distributed evolutionary algorithms and their models: A survey of the state-of-the-art
Gong, Y.-J., Chen, W.-N., Zhan, Z.-H., Zhang, J., Li, Y., Zhang, Q., & Li, J.-J. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.04.061 · Cited by 393
Distributed Algorithms for Topic Models
Newman, D. J., Asuncion, A., Smyth, P., & Welling, M. (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755845 · Cited by 393
Gönen, M. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts360 · Cited by 383
CNTK: Microsoft's Open-Source Deep-Learning Toolkit
Seide, F., & Agarwal, A. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2945397 · Cited by 382
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369
Benchmarking State-of-the-Art Deep Learning Software Tools
Shi, S., Wang, Q., Xu, P., & Chu, X. (2016) · 2016 7th International Conference on Cloud Computing and Big Data (CCBD) · doi:10.1109/ccbd.2016.029 · Cited by 329
Principles of Transaction Processing
(2009) · Elsevier eBooks · doi:10.1016/b978-1-55860-623-4.x0001-7 · Cited by 324
Accelerating Deep Convolutional Neural Networks Using Specialized Hardware
Ovtcharov, K., Ruwase, O., Kim, J.-Y., Fowers, J., Strauß, K., & Chung, E. S. (2015) · Cited by 310
Design and evaluation of a conit-based continuous consistency model for replicated services
Yu, H., & Vahdat, A. (2002) · ACM Transactions on Computer Systems · doi:10.1145/566340.566342 · Cited by 289
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · ACM SIGPLAN Notices · doi:10.1145/2644865.2541967 · Cited by 281
A Survey of Methods for Scaling Up Inductive Algorithms
Provost, F., & Kolluri, V. (1999) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009876119989 · Cited by 277
A Reliable Effective Terascale Linear Learning System
Agarwal, A., Chapelle, O., Dudı́k, M., & Langford, J. C. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1110.4198 · Cited by 260
Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization
Shalev‐Shwartz, S., & Zhang, T. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1209.1873 · Cited by 240
A survey of methods for distributed machine learning
Peteiro-Barral, D., & Guijarro-Berdiñas, B. (2013) · Progress in Artificial Intelligence · doi:10.1007/s13748-012-0035-5 · Cited by 206
Newton’s method and its use in optimization
Polyak, B. T. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2005.06.076 · Cited by 190
Rule-based Machine Learning Methods for Functional Prediction
Weiss, S. M., & Indurkhya, N. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.199 · Cited by 183
Distributed Coordinate Descent Method for Learning with Big Data
Richtárik, P., & Takáč, M. (2013) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.1310.2059 · Cited by 182
Gossip learning with linear models on fully distributed data
Ormándi, R., Hegedűs, I., & Jelasity, M. (2012) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.2858 · Cited by 169
Strategies and Principles of Distributed Machine Learning on Big Data
Xing, E. P., Ho, Q., Xie, P., & Wei, D. (2016) · Engineering · doi:10.1016/j.eng.2016.02.008 · Cited by 162
Large Scale Distributed Data Science using Apache Spark
Shanahan, J. G., & Dai, L. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2789993 · Cited by 161
No Peek: A Survey of private distributed deep learningPreprint
Vepakomma, P., Swedish, T. B., Raskar, R., Gupta, O., & Dubey, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03288 · Cited by 143
Chen, K., & Huo, Q. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472805 · Cited by 137
On Distributed Communications Networks
Baran, P. (1962) · doi:10.7249/p2626 · Cited by 129
Mini-Batch Primal and Dual Methods for SVMsPreprint
Takáč, M., Bijral, A. S., Richtárik, P., & Srebro, N. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1303.2314 · Cited by 128
Exploiting bounded staleness to speed up Big Data analytics
Cui, H., Cipar, J., Ho, Q., Kim, J. K., Lee, S., Kumar, A., Wei, J., Dai, W., Ganger, G. R., Gibbons, P. B., et al. (2018) · doi:10.1184/r1/6475613 · Cited by 119
Distributed Learning, Communication Complexity and PrivacyPreprint
Balcan, M.-F., Blum, A. L., Fine, S., & Mansour, Y. (2012) · arXiv (Cornell University) · doi:10.1184/r1/6475568 · Cited by 119
Big data analysis using Apache Hadoop
Nandimath, J., Banerjee, E., Patil, A., Kakade, P., Vaidya, S., & Chaturvedi, D. N. (2013) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri.2013.6642536 · Cited by 106
Managed communication and consistency for fast data-parallel iterative analytics
Wei, J., Dai, W., Qiao, A., Ho, Q., Cui, H., Ganger, G. R., Gibbons, P. B., Gibson, G. A., & Xing, E. P. (2015) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2806777.2806778 · Cited by 104
Yan, Y., Gao, Y., Chen, Y., et al. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987576 · Cited by 90
Kickstarting high-performance energy-efficient manycore architectures with Epiphany
Olofsson, A., Nordström, T., & Ul-Abdin, Z. (2014) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2014.7094761 · Cited by 89
A reliable effective terascale linear learning system
AgarwalAlekh, ChapelleOlivier, DudíkMiroslav, & LangfordJohn (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2638571 · Cited by 89
Distributed data mining: a survey
Zeng, L., Li, L., Duan, L., Lu, K., Shi, Z., Wang, M., Wu, W., & Luo, P. (2012) · Information Technology and Management · doi:10.1007/s10799-012-0124-y · Cited by 85
AIOLOS: Middleware for improving mobile application performance through cyber foraging
Verbelen, T., Simoens, P., Turck, F. D., & Dhoedt, B. (2012) · Journal of Systems and Software · doi:10.1016/j.jss.2012.06.011 · Cited by 82
Self-citationDeep learning at 15PF: supervised and semi-supervised classification for scientific data
Kurth, T., Zhang, J., Nadathur, S., Racah, E., Mitliagkas, I., Patwary, M. M. A., Malas, T. B., Sundaram, N., Bhimji, W., Smorkalov, M. E., et al. (2017) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3126908.3126916 · Cited by 82
Parallel approaches to machine learning—A comprehensive survey
Upadhyaya, S. R. (2012) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2012.11.001 · Cited by 81
Scaling deep learning on GPU and knights landing clusters
You, Y., Buluç, A., & Demmel, J. W. (2017) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3126908.3126912 · Cited by 75
Machine Learning and Cloud Computing: Survey of Distributed and SaaS SolutionsPreprint
Pop, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08767 · Cited by 73
Genomic big data hitting the storage bottleneck
Papageorgiou, L., Picasi, E., Raftopoulou, S., Mantaiou, M., Megalooikonomou, V., & Vlachakis, D. (2018) · EMBnet journal · doi:10.14806/ej.24.0.910 · Cited by 63
Fuzzy-UCS: A Michigan-Style Learning Fuzzy-Classifier System for Supervised Learning
Orriols-Puig, A., Casillas, J., & Bernadó-Mansilla, E. (2008) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.925144 · Cited by 62
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis
Raicu, I., Foster, I. T., Szalay, A. S., & Turcu, G. (2006) · Cited by 44
Coninck, E. D., Bohez, S., Leroux, S., Verbelen, T., Vankeirsbilck, B., Simoens, P., & Dhoedt, B. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2018.03.032 · Cited by 29
Self-citationSERF: Efficient Scheduling for Fast Deep Neural Network Serving via Judicious Parallelism
Yan, F., Ruwase, O., He, Y., & Smirni, E. (2016) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.2016.25 · Cited by 29
(2014) · doi:10.5555/2627435.2697065
Cited by
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges
Khan, L. U., Saad, W., Han, Z., Hossain, E., & Hong, C. S. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3090430 · Cited by 892
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
Yin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638
Holistic Network Virtualization and Pervasive Network Intelligence for 6G
Shen, X., Gao, J., Wu, W., Li, M., Zhou, C., & Zhuang, W. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3135829 · Cited by 392
6G Wireless Systems: A Vision, Architectural Elements, and Future Directions
Khan, L. U., Yaqoob, I., Imran, M., Han, Z., & Hong, C. S. (2020) · IEEE Access · doi:10.1109/access.2020.3015289 · Cited by 384
Metadata
- Authors
- Verbraeken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., & Rellermeyer, J. S.
- Venue
- ACM Computing Surveys
- Published
- 2020-03-20
- Volume
- 53
- Issue
- 2
- Pages
- 1-33
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2996075766
- Cited by
- 872
- References
- 125
- FWCI
- 41.39
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.