Survey and Benchmarking of Machine Learning Accelerators
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Advances in multicore processors and accelerators have opened the flood gates to greater exploration and application of machine learning techniques to a variety of applications. These advances, along with breakdowns of several trends including Moore's Law, have prompted an explosion of processors and accelerators that promise even greater computational and machine learning capabilities. These processors and accelerators are coming in many forms, from CPUs and GPUs to ASICs, FPGAs, and dataflow accelerators. This paper surveys the current state of these processors and accelerators that have been publicly announced with performance and power consumption numbers. The performance and power values are plotted on a scatter graph and a number of dimensions and observations from the trends on this plot are discussed and analyzed. For instance, there are interesting trends in the plot regarding power consumption, numerical precision, and inference versus training. We then select and benchmark two commercially available low size, weight, and power (SWaP) accelerators as these processors are the most interesting for embedded and mobile machine learning inference applications that are most applicable to the DoD and other SWaP constrained users. We determine how they actually perform with real-world images and neural network models, compare those results to the reported performance and power consumption values and evaluate them against an Intel CPU that is used in some embedded applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 2×
“Conversely, pooling, dropout, softmax, and recurrent/skip connection layers are not computationally intensive since these types of layers stipulate datapaths for weight and data operands. including integer representations, have been shown to be reasonably effective for inference [7], [8].”
BackgroundEyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247 · cited 1×
“• MIT Eyeriss chip 〈Eyeriss〉 [7], [15], [16] is a research chip from Vivienne Sze’s group in MIT CSAIL.”
BackgroundWilliams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · cited 1×
“Manyhardware manufacturers, faced with limitations in fabrication processes, have been able to exploit the fact that machine-learning algorithms such as neural networks can perform well even when using limited or mixed precision [8], [13] representation of activation functions, weights, and biases.”
BackgroundMicikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179 · cited 2×
“Conversely, pooling, dropout, softmax, and recurrent/skip connection layers are not computationally intensive since these types of layers stipulate datapaths for weight and data operands. including integer representations, have been shown to be reasonably effective for inference [7], [8].”
BackgroundAn Analysis of Deep Neural Network Models for Practical Applications
Canziani, A., Paszke, A., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1605.07678 · Cited by 969 · cited 1×
“Overall, the most emphasis of computational capability for machine learning is on DNN and CNNs because they are quite computationally intensive [6], with the fully connected and convolutional layers being the most computationally intense.”
BackgroundESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021745 · Cited by 591 · cited 1×
“…an abbreviation of their chip type: (cid:104) Zynq-020 (cid:105) int1 [36], int2 [37], int8 [38]; Zynq-060 (cid:105) int16 accumulator/int12 result [39], (cid:104) ZCU102 int16 [40], (cid:104) Stratix-V (cid:105) int32 [41], (cid:104) ArriaGX1150 (cid:105) int16 ac-cumulator/int8 result [42],…”
[DL] A Survey of FPGA-based Neural Network Inference Accelerators
Guo, K., Zeng, S., Yu, J., Wang, Y., & Yang, H. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3289185 · Cited by 292
Cited by
Ising machines as hardware solvers of combinatorial optimization problems
Mohseni, N., McMahon, P. L., & Byrnes, T. (2022) · Nature Reviews Physics · doi:10.1038/s42254-022-00440-8 · Cited by 543
The Computational Limits of Deep Learning
Thompson, N. C., Greenewald, K., Lee, K., & Manso, G. F. (2020) · arXiv · doi:10.48550/arxiv.2007.05558 · Cited by 538
“9× in compute per watt from 2017 to 2020 [75, 76, 77, 73, 74].”
BackgroundCloudscout: a deep neural network for on-board cloud detection on hyperspectral images
Giuffrida, G., Diana, L., de Gioia, F., Benelli, G., Meoni, G., Donati, M., & Fanucci, L. (2020) · Remote Sensing · doi:10.3390/rs12142205 · Cited by 176
“The COTS accelerators are easily classifiable by their processors [14,15]: VPU, Tensor Processing Unit (TPU), the most known GPU and Field-Programmable Gate Array (FPGA).”
BackgroundMethodInfluentialAI and ML Accelerator Survey and Trends
Reuther, A., Michaleas, P., Jones, M., Gadepally, V., Samsi, S., & Kepner, J. (2022) · 2022 IEEE High Performance Extreme Computing Conference (HPEC) · doi:10.1109/hpec55821.2022.9926331 · Cited by 109
Self-citationExploring Edge TPU for Network Intrusion Detection in IoT
Hosseininoorbin, S., Layeghy, S., Sarhan, M., Jurdak, R., & Portmann, M. (2021) · arXiv · doi:10.48550/arxiv.2103.16295 · Cited by 45
Metadata
- Authors
- Reuther, A., Michaleas, P., Jones, M., Gadepally, V., Samsi, S., & Kepner, J.
- Venue
- 2019 IEEE High Performance Extreme Computing Conference (HPEC)
- Publisher
- IEEE
- Published
- 2019-09
- Pages
- 1-9
- OpenAlex
- W2970959587
- Cited by
- 175
- References
- 11
- FWCI
- 18.93
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.