The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools
Abstract
Deep Learning (DL) has had an immense success in the recent past, leading to state-of-the-art results in various domains, such as image recognition and natural language processing. One of the reasons for this success is the increasing size of DL models and the proliferation of vast amounts of training data being available. To keep on improving the performance of DL, increasing the scalability of DL systems is necessary. In this survey, we perform a broad and thorough investigation on challenges, techniques and tools for scalable DL on distributed infrastructures. This incorporates infrastructures for DL, methods for parallel DL training, multi-tenant resource scheduling, and the management of training and model data. Further, we analyze and compare 11 current open-source DL frameworks and tools and investigate which of the techniques are commonly implemented in practice. Finally, we highlight future research trends in DL systems that deserve further research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“Version of Record will be published in ACM Computing Surveys. 1 INTRODUCTION Deep Learning (DL) has recently gained a lot of attention due to its superior performance in tasks like speech recognition [65,69], optical character recognition [20], and object detection [95]. The application of DL poses a tremendous potent…”
Searle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
Research works printed but not linked: 162
- Abadi Martín, 2016, Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’16)
- 10.1145/2976749.2978318 · DOI · Paper page
- 10.5555/2627435.2638571 · doi:10.5555/2627435.2638571
- Akiba Takuya, 2017, Proceedings of the Workshop on ML Systems in the 31st Annual Conference on Neural Information Processing Systems (NIPS’17)
- Alistarh Dan, 2017, QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems 30. Curran Associates
- 10.1145/1721654.1721672 · DOI · Paper page
- 10.1109/TPDS.2017.2752706 · DOI
- Baker Bowen, 2017, Proceedings of the International Conference on Learning Representations.
- Barroso Luiz Andre, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
- Tal Ben-Nun and Torsten Hoefler. 2018. Demystifying parallel and distributed deep learning: An in-depth concurrency analysis. Retrieved from http://arxiv.org/abs/1802.09941. Tal Ben-Nun and Torsten Hoefler. 2018. Demystifying parallel and distributed deep learning: An in-depth concurrency analysis. Retrieved from http://arxiv.org/abs/1802.09941. · arXiv
- Bergstra James, 2011, Proceedings of the BigLearning Workshop (NIPS’11), 3
- Bergstra James, 2012, Random search for hyper-parameter optimization, J. Mach. Learn. Res., 13, 1
- 10.1147/JRD.2017.2716578 · DOI
- Boag Scott, 2017, Proceedings of the Workshop on ML Systems (NIPS’17)
- Bonawitz Keith, 2019, Proceedings of the Conference on Systems and Machine Learning (SysML’19)
- 10.1145/3219819.3219861 · DOI · Paper page
- 10.1145/1961189.1961199 · DOI · Paper page
- Chen K., 2016, Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’16)
- Chen Tianqi, 2015, Proceedings of Workshop on Machine Learning Systems (LearningSys’15)
- 10.1109/ACCESS.2014.2325029 · DOI · Paper page
- Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan Cohen John Tran Bryan Catanzaro and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. Retrieved from http://arxiv.org/abs/1410.0759. Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan Cohen John Tran Bryan Catanzaro and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. Retrieved from http://arxiv.org/abs/1410.0759. · arXiv
- Chilimbi Trishul, 2014, Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14)
- Cipar James, 2013, Proceedings of the 14th Workshop on Hot Topics in Operating Systems (HotOS’13)
- 10.1162/NECO_a_00052 · DOI · Paper page
- Coates Adam, 2013, Proceedings of the 30th International Conference on International Conference on Machine Learning, 28
- Crankshaw Daniel, 2017, Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI’17)
- Cui Henggang, Proceedings of the USENIX Conference on USENIX Annual Technical Conference (USENIX ATC’14)
- Cui Henggang, Proceedings of the 11th European Conference on Computer Systems (EuroSys’16)
- Dai Wei, Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI’15)
- Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V. Le etal 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems. MIT Press 1223--1231. Jeffrey Dean Greg Corrado Rajat Monga Kai Chen Matthieu Devin Mark Mao Andrew Senior Paul Tucker Ke Yang Quoc V. Le et al. 2012. Large scale distributed deep networks. In Advances in Neural Information Processing Systems. MIT Press 1223--1231.
- 10.1145/1327452.1327492 · DOI · Paper page
- 10.1017/atsip.2013.9 · DOI · Paper page
- 10.1515/pralin-2017-0036 · DOI
- Duchi John, 2011, Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12 (July
- 10.1007/s10278-017-9965-6 · DOI · Paper page
- Farabet C., 2011, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11)
- Fischer Volker, Advances in Neural Information Processing Systems 31
- Frank Eibe, Weka-A Machine Learning Workbench for Data Mining, 10.1007/978-0-387-09823-4_66 · DOI · Paper page
- Ghodsi Ali, 2011, Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI’11)
- Goodfellow Ian, Advances in Neural Information Processing Systems
- Graves Alex, 2014, International Conference on Machine Learning. MIT Press, 1764--1772
- Gujarati Arpan, Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference (Middleware’17)
- Gupta Suyog, 2015, Proceedings of the 32nd International Conference on International Conference on Machine Learning, 37
- 10.1109/MIS.2009.36 · DOI · Paper page
- Harlap Aaron, 2018, Gibbons
- Harlap Aaron, Proceedings of the Conference on Systems and Machine Learning (SysML’18)
- Harlap Aaron, Proceedings of the 12th European Conference on Computer Systems (EuroSys’17)
- Hashemi Sayed Hadi, Proceedings of the Conference on Systems and Machine Learning (SysML’19)
- Hauswald J., Proceedings of the ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA’15)
- Hazelwood K., 2018, Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA’18)
- Hestness Joel, 2017, Yang Yang, and Yanqi Zhou.
- Hindman Benjamin, 2011, Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI’11)
- G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 5786 (2006) 504--507. DOI:https://doi.org/10.1126/science.1127647 arXiv:http://science.sciencemag.org/content/313/5786/504.full.pdf G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 5786 (2006) 504--507. DOI:https://doi.org/10.1126/science.1127647 arXiv:http://science.sciencemag.org/content/313/5786/504.full.pdf · DOI · Paper page
- 10.1145/2500887 · DOI
- Yanping Huang Yonglong Cheng Dehao Chen HyoukJoong Lee Jiquan Ngiam Quoc V. Le and Zhifeng Chen. 2018. GPipe: Efficient training of giant neural networks using pipeline parallelism. Retrieved from http://arxiv.org/abs/1811.06965. Yanping Huang Yonglong Cheng Dehao Chen HyoukJoong Lee Jiquan Ngiam Quoc V. Le and Zhifeng Chen. 2018. GPipe: Efficient training of giant neural networks using pipeline parallelism. Retrieved from http://arxiv.org/abs/1811.06965. · arXiv
- 10.1145/3187009.3177734 · doi:10.1145/3187009.3177734
- 10.1113/jphysiol.1962.sp006837 · DOI · Paper page
- Hutter Frank, 2014, Proceedings of the 31st International Conference on International Conference on Machine Learning, 32
- 10.1109/CVPR.2016.284 · DOI · Paper page
- 10.1145/2897824.2925974 · DOI · Paper page
- Ishakian V., 2018, Proceedings of the IEEE International Conference on Cloud Engineering (IC2E’18)
- 10.1145/3286490.3286561 · DOI
- Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Junjie Qian Wencong Xiao and Fan Yang. 2019. Analysis of large-scale multi-tenant GPU clusters for DNN training workloads. arXiv preprint arXiv:1901.05758. Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Junjie Qian Wencong Xiao and Fan Yang. 2019. Analysis of large-scale multi-tenant GPU clusters for DNN training workloads. arXiv preprint arXiv:1901.05758. · arXiv
- 10.1145/3190508.3190530 · DOI
- 10.1145/2647868.2654889 · DOI · Paper page
- Jia Zhihao, 2018, Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), 80
- Jia Zhihao, 2018, Beyond data and model parallelism for deep neural networks. CoRR abs/1807.05358 · arXiv
- 10.1145/3035918.3035933 · DOI · Paper page
- 10.1162/tacl_a_00065 · DOI · Paper page
- Johnson Rie, 2013, Proceedings of the 26th International Conference on Neural Information Processing Systems, 1
- Jouppi N. P., Proceedings of the ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA’17)
- Karkus Peter, 2017, QMDP-net: Deep learning for planning under partial observability, Advances in Neural Information Processing Systems, 30, I
- Kim Jin Kyu, Proceedings of the 11th European Conference on Computer Systems (EuroSys’16)
- Koliousis Alexandros, 2019, Pietzuch
- Konec̆ný Jakub, 2016, Proceedings of the NIPS Workshop on Private Multi-Party Machine Learning.
- Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. Retrieved from http://arxiv.org/abs/1404.5997. Alex Krizhevsky. 2014. One weird trick for parallelizing convolutional neural networks. Retrieved from http://arxiv.org/abs/1404.5997. · arXiv
- Krizhevsky Alex, Proceedings of the 25th International Conference on Neural Information Processing Systems, 1
- 10.1145/3035918.3054775 · DOI
- 10.1145/1815961.1816021 · DOI · Paper page
- Lee Lee Yun Seong, 2016, Proceedings of International Conference on Machine Learning (ICML’16)
- 10.1177/0278364914549607 · DOI · Paper page
- Li Ang, 2019, Jieyang Chen, Jiajia Li, Xu Liu, Nathan R. Tallent, and Kevin J. Barker.
- 10.1145/2741948.2741965 · DOI
- 10.5555/2685048.2685095 · doi:10.5555/2685048.2685095
- 10.1016/j.future.2017.02.006 · DOI · Paper page
- 10.1145/3187009.3177737 · DOI
- Lian Xiangru, Advances in Neural Information Processing Systems
- Lin Yujun, 2018, Proceedings of the International Conference on Learning Representations.
- Liu Jiayi, 2018, Proceedings of the Deep Learning Day at SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’18)
- Mai Luo, 2015, Proceedings of the 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud’15)
- 10.1145/1807167.1807184 · DOI · Paper page
- Dominic Masters and Carlo Luschi. 2018. Revisiting small batch training for deep neural networks. Retrieved from arxiv:1804.07612 http://arxiv.org/abs/1804.07612. Dominic Masters and Carlo Luschi. 2018. Revisiting small batch training for deep neural networks. Retrieved from arxiv:1804.07612 http://arxiv.org/abs/1804.07612. · arXiv
- 10.1145/3154842.3154843 · doi:10.1145/3154842.3154843
- McMahan H. Brendan, 2017, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS’17)
- Meng Xiangrui, 2016, MLlib: Machine learning in apache spark, J. Mach. Learn. Res., 17, 1
- Miao H., 2017, Proceedings of the IEEE 33rd International Conference on Data Engineering (ICDE’17)
- Mirhoseini Azalia, 2018, Proceedings of the International Conference on Learning Representations.
- Mirhoseini Azalia, 2017, Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.), 70
- Mnih Volodymyr, Proceedings of the 29th International Conference on Machine Learning (ICML’12)
- Moritz Philipp, Proceedings of International Conference on Learning Representations. arXiv preprint arXiv:1511
- Moritz Philipp, 2018, Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI’18)
- Narayanan Deepak, 2018, Proceedings of the NIPS Workshop on Systems for Machine Learning.
- Nilsson Nils, The Quest for Artificial Intelligence: A History of Ideas and Achievements, 10.1017/CBO9780511819346 · DOI · Paper page
- Noel Cyprien, 2014, Proceedings of the NIPS Workshop on Distributed Machine Learning and Matrix Computations.
- 10.1145/2733373.2807410 · DOI
- 10.1145/3286490.3286562 · doi:10.1145/3286490.3286562
- Park Jay H., 2019, Noh
- Paszke Adam, 2017, Proceedings of the Conference and Workshop on Neural Information Processing Systems (NIPS-W’17)
- Pedregosa Fabian, 2011, Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (Oct.
- 10.1145/3190508.3190517 · DOI · Paper page
- Pinto Christian, 2018, Hoard: A distributed data caching system to accelerate deep-learning training on the cloud.
- 10.1145/3234150 · DOI · Paper page
- Qiao Aurick, Proceedings of the USENIX Annual Technical Conference (USENIX ATC 18)
- 10.1145/3135974.3135994 · DOI
- Recht Benjamin, 2011, Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in Neural Information Processing Systems
- 10.1145/3199605 · DOI
- Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. Retrieved from http://arxiv.org/abs/1609.04747. Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. Retrieved from http://arxiv.org/abs/1609.04747. · arXiv
- Theo Ryffel Andrew Trask Morten Dahl Bobby Wagner Jason Mancuso Daniel Rueckert and Jonathan Passerat-Palmbach. 2018. A generic framework for privacy preserving deep learning. Retrieved from http://arxiv.org/abs/1811.04017. Theo Ryffel Andrew Trask Morten Dahl Bobby Wagner Jason Mancuso Daniel Rueckert and Jonathan Passerat-Palmbach. 2018. A generic framework for privacy preserving deep learning. Retrieved from http://arxiv.org/abs/1811.04017. · arXiv
- Christopher De Sa Megan Leszczynski Jian Zhang Alana Marzoev Christopher R. Aberger Kunle Olukotun and Christopher Ré. 2018. High-accuracy low-precision training. Retrieved from http://arxiv.org/abs/1803.03383. Christopher De Sa Megan Leszczynski Jian Zhang Alana Marzoev Christopher R. Aberger Kunle Olukotun and Christopher Ré. 2018. High-accuracy low-precision training. Retrieved from http://arxiv.org/abs/1803.03383. · arXiv
- 10.1016/j.neunet.2014.09.003 · DOI · Paper page
- 10.21437/Interspeech.2014-274 · DOI · Paper page
- Sergeev Alexander, 2018, Horovod: Fast and easy distributed deep learning in TensorFlow.
- Shazeer Noam, 2018, Mesh-tensorflow: Deep learning for supercomputers. In Advances in Neural Information Processing Systems
- 10.14778/1920841.1920931 · DOI · Paper page
- Socher Richard, Advances in Neural Information Processing Systems 26
- 10.1145/2806777.2806945 · DOI
- 10.21437/Interspeech.2015-354 · DOI · Paper page
- Sainbayar Sukhbaatar and Rob Fergus. 2014. Learning from noisy labels with deep neural networks. Retrieved from http://arxiv.org/abs/1406.2080. Sainbayar Sukhbaatar and Rob Fergus. 2014. Learning from noisy labels with deep neural networks. Retrieved from http://arxiv.org/abs/1406.2080. · arXiv
- 10.1109/CVPR.2016.308 · DOI · Paper page
- Tao Zeyi, 2018, Proceedings of the USENIX Workshop on Hot Topics in Edge Computing (HotEdge'18)
- Tokui Seiya, 2015, Proceedings of Workshop on Machine Learning Systems (LearningSys) in The 29th Annual Conference on Neural Information Processing Systems (NIPS’15)
- 10.1145/79173.79181 · DOI
- Vanhoucke Vincent, Proceedings of the Deep Learning and Unsupervised Feature Learning Workshop (NIPS’11)
- 10.1145/2939502.2939516 · DOI
- Abhinav Vishnu Charles Siegel and Jeffrey Daily. 2016. Distributed TensorFlow with MPI. Retrieved from http://arxiv.org/abs/1603.02339. Abhinav Vishnu Charles Siegel and Jeffrey Daily. 2016. Distributed TensorFlow with MPI. Retrieved from http://arxiv.org/abs/1603.02339. · arXiv
- Wang C., 2017, DLAU: A scalable deep-learning accelerator unit on FPGA, IEEE Trans. Comput.-Aid. Design Integr. Circ. Syst., 36, 3
- Wang Minjie, 2014, Proceedings of the NIPS Workshop on Distributed Machine Learning and Matrix Computations.
- 10.1145/2637166.2637231 · DOI
- 10.1145/3274808.3274828 · doi:10.1145/3274808.3274828
- 10.1145/3003665.3003669 · DOI · Paper page
- 10.1145/2987550.2987586 · DOI
- Wei Jinliang, Proceedings of the 6th ACM Symposium on Cloud Computing (SoCC’15)
- 10.1145/2723372.2742793 · DOI
- Wen Wei, 2017, Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in Neural Information Processing Systems
- Yonghui Wu Mike Schuster Zhifeng Chen Quoc V. Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey Jeff Klingner Apurva Shah Melvin Johnson Xiaobing Liu Lukasz Kaiser Stephan Gouws Yoshikiyo Kato Taku Kudo Hideto Kazawa Keith Stevens George Kurian Nishant Patil Wei Wang Cliff Young Jason Smith Jason Riesa Alex Rudnick Oriol Vinyals Greg Corrado Macduff Hughes and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. Retrieved from http://arxiv.org/abs/1609.08144. Yonghui Wu Mike Schuster Zhifeng Chen Quoc V. Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey Jeff Klingner Apurva Shah Melvin Johnson Xiaobing Liu Lukasz Kaiser Stephan Gouws Yoshikiyo Kato Taku Kudo Hideto Kazawa Keith Stevens George Kurian Nishant Patil Wei Wang Cliff Young Jason Smith Jason Riesa Alex Rudnick Oriol Vinyals Greg Corrado Macduff Hughes and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. Retrieved from http://arxiv.org/abs/1609.08144. · arXiv
- Wu Zonghan, 2019, Yu
- Xiao Tong, 2015, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15)
- Xiao Wencong, 2018, Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI’18)
- 10.1109/TBDATA.2015.2472014 · DOI · Paper page
- 10.1145/2783258.2783270 · DOI
- 10.1145/3190508.3190551 · DOI
- Zaharia Matei, Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI’12). USENIX
- 10.1145/2966986.2967011 · DOI · Paper page
- 10.1145/2684746.2689060 · DOI · Paper page
- Zhang C., 2018, Proceedings of the IEEE 38th International Conference on Distributed Computing Systems (ICDCS’18)
- Zhang H., 2016, Proceedings of the IEEE 16th International Conference on Data Mining (ICDM’16)
- Zhang Haoyu, Proceedings of the Symposium on Cloud Computing (SoCC’17)
- Zhang Hao, Proceedings of the USENIX Conference on Usenix Annual Technical Conference (USENIX ATC’17)
- Zhao Huasha, 1972, Proceedings of the SIAM International Conference on Data Mining. 785--793
- 10.1109/CVPR.2016.485 · DOI · Paper page
- Shuchang Zhou Zekun Ni Xinyu Zhou He Wen Yuxin Wu and Yuheng Zou. 2016. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. Retrieved from http://arxiv.org/abs/1606.06160. Shuchang Zhou Zekun Ni Xinyu Zhou He Wen Yuxin Wu and Yuheng Zou. 2016. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. Retrieved from http://arxiv.org/abs/1606.06160. · arXiv
- Zoph Barret, 2017, Proceedings of the International Conference on Learning Representations.
Cited by
Deep Learning for Time Series Forecasting: A Survey
Torres, J. F., Hadjout, D., Sebaa, A., Martínez-Álvarez, F., & Troncoso, A. (2021) · Big Data · doi:10.1089/big.2020.0159 · Cited by 818
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects
Massaoudi, M., Abu-Rub, H., Refaat, S. S., Chihi, I., & Oueslati, F. S. (2021) · IEEE Access · doi:10.1109/access.2021.3071269 · Cited by 256
PySyft: A Library for Easy Federated Learning
Ziller, A., Trask, A., Lopardo, A., Szymkow, B., Wagner, B., Bluemke, E., Nounahon, J.-M., Passerat-Palmbach, J., Prakash, K., Rose, N., et al. (2021) · Studies in Computational Intelligence · doi:10.1007/978-3-030-70604-3_5 · Cited by 246
Deepfakes: current and future trends
Gambín, Á. F., Yazidi, A., Vasilakos, A., Haugerud, H., & Djenouri, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10679-x · Cited by 145
Metadata
- Authors
- Mayer, R., & Jacobsen, H.-A.
- Venue
- ACM Computing Surveys
- Published
- 2020-02-06
- Volume
- 53
- Issue
- 1
- Pages
- 1-37
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3004495293
- Cited by
- 192
- References
- 28
- FWCI
- 25.80
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools. https://app.cochat.ai/papers/10.1145/3363554
- The paper
- Mayer, R., & Jacobsen, H.-A. (2021). Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools. ACM Computing Surveys. https://doi.org/10.1145/3363554
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.