The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks
Abstract
Convolutional Neural Networks (CNNs) have gained popularity in many computer vision applications such as image classification, face detection, and video analysis, because of their ability to train and classify with high accuracy. Due to multiple convolution and fully-connected layers that are compute-/memory-intensive, it is difficult to perform real-time classification with low power consumption on today?s computing systems. FPGAs have been widely explored as hardware accelerators for CNNs because of their reconfigurability and energy efficiency, as well as fast turn-around-time, especially with high-level synthesis methodologies. Previous FPGA-based CNN accelerators, however, typically implemented generic accelerators agnostic to the CNN configuration, where the reconfigurable capabilities of FPGAs are not fully leveraged to maximize the overall system throughput. In this work, we present a systematic design space exploration methodology to maximize the throughput of an OpenCL-based FPGA accelerator for a given CNN model, considering the FPGA resource constraints such as on-chip memory, registers, computational resources and external memory bandwidth. The proposed methodology is demonstrated by optimizing two representative large-scale CNNs, AlexNet and VGG, on two Altera Stratix-V FPGA platforms, DE5-Net and P395-D8 boards, which have different hardware resources. We achieve a peak performance of 136.5 GOPS for convolution operation, and 117.8 GOPS for the entire VGG network that performs ImageNet classification on P395-D8 board.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 1×
“Keywords FPGA, OpenCL, Convolutional Neural Networks, Optimization.”
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Advances in Neural Information Processing Systems 37
(2024) · doi:10.52202/079017 · Cited by 2,791
Recurrent Convolutional Neural Networks for Text Classification
Lai, S., Xu, L., Liu, K., & Zhao, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9513 · Cited by 2,280
Convolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
Research works printed but not linked: 2
- Khronos OpenCL Working Group . The OpenCL Specification, version 1.1.44 , 2011 . Khronos OpenCL Working Group. The OpenCL Specification, version 1.1.44, 2011.
- 10.1145/2664666.2664670 · doi:10.1145/2664666.2664670
Cited by
FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
Umuroglu, Y., Fraser, N. J., Gambardella, G., Blott, M., Leong, P., Jahre, M., & Vissers, K. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021744 · Cited by 1,189
Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network
Sharma, H., Park, J., Suda, N., Lai, L., Chau, B., Kim, J. K., Chandra, V., & Esmaeilzadeh, H. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00069 · Cited by 543
Self-citationA Configurable Cloud-Scale DNN Processor for Real-Time AI
Fowers, J., Ovtcharov, K., Papamichael, M., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., Adams, L., Ghandi, M., et al. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00012 · Cited by 520
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review
Shawahna, A., Sait, S. M., & El-Maleh, A. (2019) · IEEE Access · doi:10.1109/access.2018.2890150 · Cited by 496
Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?
Nurvitadhi, E., Venkatesh, G., Sim, J., Marr, D., Huang, R., Ong Gee Hock, J., Liew, Y. T., Srivatsan, K., Moss, D., Subhaschandra, S., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021740 · Cited by 467
A survey of FPGA-based accelerators for convolutional neural networks
Mittal, S. (2020) · Neural Computing and Applications · doi:10.1007/s00521-018-3761-1 · Cited by 429
Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs
Wei, X., Yu, C. H., Zhang, P., Chen, Y., Wang, Y., Hu, H., Liang, Y., & Cong, J. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062207 · Cited by 426
Optimizing Loop Operation and Dataflow in FPGA Acceleration of Deep Convolutional Neural Networks
Ma, Y., Cao, Y., Vrudhula, S., & Seo, J.-S. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021736 · Cited by 389
Self-citationFrom high-level deep neural models to FPGAs
Sharma, H., Park, J., Mahajan, D., Amaro, E., Kim, J. K., Shao, C., Mishra, A., & Esmaeilzadeh, H. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783720 · Cited by 365
A Survey of Neuromorphic Computing and Neural Networks in Hardware
Schuman, C. D., Potok, T. E., Patton, R. M., Birdwell, J. D., Dean, M. E., Rose, G. S., & Plank, J. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06963 · Cited by 352
Computer vision algorithms and hardware implementations: A survey
Feng, X., Jiang, Y., Yang, X., Du, M., & Li, X. (2019) · Integration · doi:10.1016/j.vlsi.2019.07.005 · Cited by 349
A survey on deep learning in medicine: Why, how and when?
Piccialli, F., Somma, V. D., Giampaolo, F., Cuomo, S., & Fortino, G. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.09.006 · Cited by 348
DNNBuilder: an automated tool for building high-performance DNN hardware accelerators for FPGAs
Zhang, X., Wang, J., Zhu, C., Lin, Y., Xiong, J., Hwu, W.-M., & Chen, D. (2018) · Proceedings of the International Conference on Computer-Aided Design · doi:10.1145/3240765.3240801 · Cited by 345
Optimizing the Convolution Operation to Accelerate Deep Neural Networks on FPGA
Ma, Y., Cao, Y., Vrudhula, S., & Seo, J.-S. (2018) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2815603 · Cited by 343
Self-citationGuan, Y., Liang, H., Xu, N., Wang, W., Shi, S., Chen, X., Sun, G., Zhang, W., & Cong, J. (2017) · 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.25 · Cited by 308
Caffeine: towards uniformed representation and acceleration for deep convolutional neural networks
Zhang, C., Fang, Z., Zhou, P., Pan, P., & Cong, J. (2016) · Proceedings of the 35th International Conference on Computer-Aided Design · doi:10.1145/2966986.2967011 · Cited by 299
A high performance FPGA-based accelerator for large-scale convolutional neural networks
Li, H., Fan, X., Jiao, L., Cao, W., Zhou, X., & Wang, L. (2016) · 2016 26th International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2016.7577308 · Cited by 299
FP-BNN: Binarized neural network on FPGA
Liang, S., Yin, S., Liu, L., Luk, W., & Wei, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.09.046 · Cited by 298
[DL] A Survey of FPGA-based Neural Network Inference Accelerators
Guo, K., Zeng, S., Yu, J., Wang, Y., & Yang, H. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3289185 · Cited by 292
Maximizing CNN Accelerator Efficiency Through Resource Partitioning
Shen, Y., Ferdman, M., & Milder, P. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080221 · Cited by 284
Caffeine: Toward Uniformed Representation and Acceleration for Deep Convolutional Neural Networks
Zhang, C., Sun, G., Fang, Z., Zhou, P., Pan, P., & Cong, J. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2785257 · Cited by 270
“To address the above limitations, two of the latest studies [23], [24] started implementing the entire CNN on an FPGA.”
BackgroundMethodDefect detection in printed circuit boards using you-only-look-once convolutional neural networks
Adibhatla, V. A., Chih, H.-C., Hsu, C.-C., Cheng, J., Abbod, M. F., & Shieh, J.-S. (2020) · Electronics · doi:10.3390/electronics9091547 · Cited by 179
fpgaConvNet: Mapping regular and irregular convolutional neural networks on FPGAs
Venieris, S. I., & Bouganis, C.-S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2844093 · Cited by 161
“throughput, latency, or multiobjective criteria, as opposed to works [47], [49], [50], [52], [57] which aim solely for throughput maximization.”
BackgroundDeep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions
Chen, Y., Zheng, B., Zhang, Z., Wang, Q., Shen, C., & Zhang, Q. (2021) · ACM Computing Surveys · doi:10.1145/3398209 · Cited by 159
“Suda et al. [165] presented an accelerator using OpenCL framework.”
MethodCharacterizing the deployment of deep neural networks on commercial edge devices
Hadidi, R., Cao, J., Xie, Y., Asgari, B., Krishna, T., & Kim, H. (2019) · 2019 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc47752.2019.9041955 · Cited by 118
Embedded Intelligence on FPGA: Survey, Applications and Challenges
Seng, K. P., Lee, P. J., & Ang, L. M. (2021) · Electronics · doi:10.3390/electronics10080895 · Cited by 105
A Survey and Taxonomy of FPGA-based Deep Learning Accelerators
Blaiech, A. G., Ben Khalifa, K., Valderrama, C., Fernandes, M. A. C., & Bedoui, M. H. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.01.007 · Cited by 88
Designing efficient accelerator of depthwise separable convolutional neural network on FPGA
Ding, W., Huang, Z., Huang, Z., Tian, L., Wang, H., & Feng, S. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2018.12.008 · Cited by 85
Li, S., Luo, Y., Sun, K., Yadav, N., & Choi, K. K. (2020) · IEEE Access · doi:10.1109/access.2020.3000009 · Cited by 75
MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores
Kao, S.-C., & Krishna, T. (2022) · 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00065 · Cited by 58
“These include manual-designed mapping search [58], [84], heuristic-based mapping search [68], [89], [101], [106] and optimization/ML methods [31], [35], [43], [44], [91].”
An Overview of FPGA Based Deep Learning Accelerators: Challenges and Opportunities
Wang, T., Wang, C., Zhou, X., & Chen, H. (2019) · 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc/smartcity/dss.2019.00229 · Cited by 53
Enabling deep learning at the IoT edge
Lai, L., & Suda, N. (2018) · Proceedings of the International Conference on Computer-Aided Design · doi:10.1145/3240765.3243473 · Cited by 49
Self-citationLei, P., Liang, J., Guan, Z., Wang, J., & Zheng, T. (2019) · IEEE Access · doi:10.1109/access.2019.2926381 · Cited by 44
A Ubiquitous Machine Learning Accelerator with Automatic Parallelization on FPGA
Wang, C., Gong, L., Li, X., & Zhou, X. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2990924 · Cited by 37
“For the neural networks, a number of literatures have been devoted [4], [5], [38], [39], [40], [41], [42].”
BackgroundA Fast and Scalable Architecture to Run Convolutional Neural Networks in Low Density FPGAs
Véstias, M. P., Duarte, R. P., de Sousa, J. T., & Neto, H. C. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.103136 · Cited by 33
Ghaffari, A., & Savaria, Y. (2020) · Electronics · doi:10.3390/electronics9122200 · Cited by 31
Boosting Convolutional Neural Networks Performance Based on FPGA Accelerator
Al-Shamma, O., Fadhel, M. A., Hameed, R. A., Alzubaidi, L., & Zhang, J. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-16657-1_47 · Cited by 27
“Convolutional Neural Networks (CNNs) [ 1 ] is one of these techniques that become the actual standard technique due to its providing high near-human accuracy in several speech recognition [2] and machine vision (e.g., segmentation [3], detection [4], and classification [5]) applications.”
BackgroundDeep Neural Network Model and FPGA Accelerator Co-Design: Opportunities and Challenges
Hao, C., & Chen, D. (2018) · 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT) · doi:10.1109/icsict.2018.8564956 · Cited by 19
“Other works such as [8] [12] [13] also attained high quality DNN implementations by exploiting partial co-design.”
BackgroundMethodCoNNa–Hardware accelerator for compressed convolutional neural networks
Struharik, R. J. R., Vukobratović, B. Z., Erdeljan, A. M., & Rakanović, D. M. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.102991 · Cited by 18
A 307-fps 351.7-GOPs/W Deep Learning FPGA Accelerator for Real-Time Scene Text Recognition
Zhao, S., An, F., & Yu, H. (2019) · 2019 International Conference on Field-Programmable Technology (ICFPT) · doi:10.1109/icfpt47387.2019.00043 · Cited by 8
Metadata
- Authors
- Suda, N., Chandra, V., Dasika, G., Mohanty, A., Ma, Y., Vrudhula, S., Seo, J.-S., & Cao, Y.
- Venue
- Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
- Publisher
- ACM
- Published
- 2016-02-21
- Pages
- 16-25
- OpenAlex
- W2294282016
- Cited by
- 553
- References
- 23
- FWCI
- 66.84
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.