Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?
Abstract
Current-generation Deep Neural Networks (DNNs), such as AlexNet and VGG, rely heavily on dense floating-point matrix multiplication (GEMM), which maps well to GPUs (regular parallelism, high TFLOP/s). Because of this, GPUs are widely used for accelerating DNNs. Current FPGAs offer superior energy efficiency (Ops/Watt), but they do not offer the performance of today's GPUs on DNNs. In this paper, we look at upcoming FPGA technology advances, the rapid pace of innovation in DNN algorithms, and consider whether future high-performance FPGAs will outperform GPUs for next-generation DNNs. The upcoming Intel® 14-nm Stratix? 10 FPGAs will have thousands of hard floating-point units (DSPs) and on-chip RAMs (M20K memory blocks). They will also have high bandwidth memories (HBMs) and improved frequency (HyperFlex? core architecture). This combination of features brings FPGA raw floating point performance within striking distance of GPUs. Meanwhile, DNNs are quickly evolving. For example, recent innovations that exploit sparsity (e.g., pruning) and compact data types (e.g., 1-2 bit) result in major leaps in algorithmic efficiency. However, these innovations introduce irregular parallelism on custom data types, which are difficult for GPUs to handle but would be a great fit for FPGA's extreme customizability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
NVIDIA DRIVE PX is an AI car computer designed to enable auto-makers to focus directly on software for autonomous vehicles.
“DRIVE PX [162] is an AI car computer which was designed to enable the auto-makers to focus directly on the software for autonomous vehicles.”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974 · see citing works
Li, F., Liu, B., Wang, X., Zhang, B., & Yan, J. (2016) · arXiv · doi:10.48550/arxiv.1605.04711 · Cited by 732 · see citing works
Accelerating Deep Convolutional Networks using low-precision and sparsity
Venkatesh, G., Nurvitadhi, E., & Marr, D. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952679 · Cited by 123 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A survey of deep learning techniques for autonomous driving
Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020) · Journal of Field Robotics · doi:10.1002/rob.21918 · Cited by 1,809 · see citing works
Mikalef, P., & Gupta, M. (2021) · Information & Management · doi:10.1016/j.im.2021.103434 · Cited by 1,700 · see citing works
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“NVIDIA’ GPUs based on Turing architecture [37]; 2) Field Programmable Gate Array (FPGA)-based hardware [51],”
MethodEquivalent-accuracy accelerated neural-network training using analogue memory
Ambrogio, S., Narayanan, P., Tsai, H., Shelby, R. M., Boybat, I., di Nolfo, C., Sidler, S., Giordano, M., Bodini, M., Farinha, N. C. P., et al. (2018) · Nature · doi:10.1038/s41586-018-0180-5 · Cited by 1,151 · see citing works
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436 · see citing works
“To minimize on-chip memory requirement, the weight matrix can be stored in a dense format [110].”
BackgroundTackling Faults in the Industry 4.0 Era—A Survey of Machine-Learning Solutions and Key Aspects
Angelopoulos, A., Michailidis, E. T., Nomikos, N., Trakadas, P., Hatziefremidis, A., Voliotis, S., & Zahariadis, T. (2019) · Sensors · doi:10.3390/s20010109 · Cited by 365 · see citing works
Lazy Batching: An SLA-aware Batching System for Cloud Machine Learning Inference
Choi, Y., Kim, Y., & Rhu, M. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00049 · Cited by 69 · see citing works
Accelerating Mobile Applications at the Network Edge with Software-Programmable FPGAs
Jiang, S., He, D., Yang, C., Xu, C., Luo, G., Chen, Y., Liu, Y., & Jiang, J. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8485850 · Cited by 40 · see citing works
Ghaffari, A., & Savaria, Y. (2020) · Electronics · doi:10.3390/electronics9122200 · Cited by 31 · see citing works
CoNNa–Hardware accelerator for compressed convolutional neural networks
Struharik, R. J. R., Vukobratović, B. Z., Erdeljan, A. M., & Rakanović, D. M. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.102991 · Cited by 18 · see citing works
- Authors
- Nurvitadhi, E., Venkatesh, G., Sim, J., Marr, D., Huang, R., Ong Gee Hock, J., Liew, Y. T., Srivatsan, K., Moss, D., Subhaschandra, S., & Boudoukh, G.
- Affiliations
- Intel (United States) (US)
- Intel, Penang, Malaysia
- Intel (Malaysia) (MY)
- Intel (Israel) (IL)
- Venue
- Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
- Publisher
- ACM
- Published
- 2017-02-22
- Pages
- 5-14
- OpenAlex
- W2583383421
- Cited by
- 467
- References
- 3
- Citations by year
- 2026: 15 · 2025: 34 · 2024: 36 · 2023: 50 · 2022: 46 · 2021: 66 · 2020: 65 · 2019: 77 · 2018: 67 · 2017: 11
- FWCI
- 42.70
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- FPGA, graphics processing units, deep neural network acceleration, high-bandwidth memory, irregular parallelism, sparse neural networks, neural network pruning, energy efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
