Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Convolutional neural network (CNN) has been widely employed for image recognition because it can achieve high accuracy by emulating behavior of optic nerves in living creatures. Recently, rapid growth of modern applications based on deep learning algorithms has further improved research and implementations. Especially, various accelerators for deep CNN have been proposed based on FPGA platform because it has advantages of high performance, reconfigurability, and fast development round, etc. Although current FPGA accelerators have demonstrated better performance over generic processors, the accelerator design space has not been well exploited. One critical problem is that the computation throughput may not well match the memory bandwidth provided an FPGA platform. Consequently, existing approaches cannot achieve best performance due to under-utilization of either logic resource or memory bandwidth. At the same time, the increasing complexity and scalability of deep learning applications aggravate this problem. In order to overcome this problem, we propose an analytical design scheme using the roofline model. For any solution of a CNN design, we quantitatively analyze its computing throughput and required memory bandwidth using various optimization techniques, such as loop tiling and transformation. Then, with the help of rooine model, we can identify the solution with best performance and lowest FPGA resource requirement. As a case study, we implement a CNN accelerator on a VC707 FPGA board and compare it to previous approaches. Our implementation achieves a peak performance of 61.62 GFLOPS under 100MHz working frequency, which outperform previous approaches significantly.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another option is to use field programable gate arrays (FPGAs), which consume much less energy than GPUs while providing an intermediate computing efficiency between CPUs and GPUs13–17.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [15]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“Figure 2 shows a real-life CNN application, taken from [9].”
BackgroundMethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“The algorithm receives three 224x224 input images that are from an original 256x256 three-channel RGB image.”
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992 · cited 1×
“Convolutional neural network (CNN), a well-known deep learning architecture extended from artificial neural network, has been extensively adopted in various applications, which include video surveillance, mobile robot vision, image search engine in data centers, etc [6] [7] [8] [10] [14].”
BackgroundMinimizing Computation in Convolutional Neural Networks
Cong, J., & Xiao, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11179-7_36 · Cited by 372 · cited 1×
“In the feedforward computation perspective, a previous study [5] proved that convolution operations will occupy over 90% of the computation time.”
BackgroundSelf-citationCNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343 · cited 8×
“Among these approaches, FPGA based accelerators have attracted more and more attention of researchers because they have advantages of good performance, high energy efficiency, fast development round, and capability of reconfiguration [1] [2] [3] [6] [12] [14].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- D. Aysegul , J. Jonghoon , G. Vinayak , K. Bharadwaj , C. Alfredo , M. Berin , and C. Eugenio . Accelerating deep neural networks on mobile processor with embedded programmable logic . In NIPS 2013 . IEEE, 2013 . D. Aysegul, J. Jonghoon, G. Vinayak, K. Bharadwaj, C. Alfredo, M. Berin, and C. Eugenio. Accelerating deep neural networks on mobile processor with embedded programmable logic. In NIPS 2013. IEEE, 2013.
Cited by
Items 501–600 of 1,002
Marvel: A Data-Centric Approach for Mapping Deep Learning Operators on Spatial Accelerators
Chatarasi, P., Kwon, H., Parashar, A., Pellauer, M., Krishna, T., & Sarkar, V. (2021) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3485137 · Cited by 41
“Prior work such as Zhang et al. [56] and Ma et al. [29] focused on spatial accelerators without L1 buffers inside a PE, again limiting their mapping space formulation.”
BackgroundMethodInfluentialTETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
Gao, M., Pu, J., Yang, X., Horowitz, M., & Kozyrakis, C. (2017) · ACM SIGOPS Operating Systems Review · doi:10.1145/3093315.3037702 · Cited by 41
Accelerating Mobile Applications at the Network Edge with Software-Programmable FPGAs
Jiang, S., He, D., Yang, C., Xu, C., Luo, G., Chen, Y., Liu, Y., & Jiang, J. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8485850 · Cited by 40
“On the other hand, there has been a lot of work on accelerating complex algorithms using FPGA, such as [8], [34], [7], etc.”
MethodReconfigurability, Why It Matters in AI Tasks Processing: A Survey of Reconfigurable AI Chips
Wei, S., Lin, X., Tu, F., Wang, Y., Liu, L., & Yin, S. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3228860 · Cited by 40
Huang, H., Ni, L., Wang, K., Wang, Y., & Yu, H. (2017) · IEEE Transactions on Nanotechnology · doi:10.1109/tnano.2017.2732698 · Cited by 40
“The low-power FPGA-based acceleration on the other hand cannot achieve high throughput due to limited computation resource (processing element and memory) [5].”
BackgroundNNest: Early-Stage Design Space Exploration Tool for Neural Network Inference Accelerators
Ke, L., He, X. J., & Zhang, X. (2018) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3218603.3218647 · Cited by 40
“2 Existing NN Accelerators A recent tutorial have extensively surveyed prior work on NN inference accelerators and categorized them based on datalow [19]: 1) No Local Reuse (NLR) represents designs which do not allocate local storage in the form of register iles or registers to each MAC unit [3, 15, 20].”
BackgroundDeep Learning Acceleration with Neuron-to-Memory Transformation
Imani, M., Samragh, M., Kim, Y., Gupta, S., Koushanfar, F., & Rosing, T. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00011 · Cited by 39
“Earlier work proposed FPGAs [10–14] and ASIC designs [15– 19] to accelerate neural networks.”
MethodInfluentialSoft Error Mitigation for Deep Convolution Neural Network on FPGA Accelerators
Li, W., Ge, G., Guo, K., Chen, X., Wei, Q., Gao, Z., Wang, Y., & Yang, H. (2020) · IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas48895.2020.9073925 · Cited by 40
“, [1], [2]) have been widely studied to offer high performance with low power consumption.”
BackgroundToward Multi-FPGA Acceleration of the Neural Networks
Biookaghazadeh, S., Ravi, P. K., & Zhao, M. (2021) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3432816 · Cited by 40
“[21] observed that the accelerator design space had not been well exploited, and the computation throughput of a design may not well match the available memory bandwidth.”
BackgroundAn Ultra-Low Power Surface EMG Sensor for Wearable Biometric and Medical Applications
Wu, Y.-D., Ruan, S.-J., & Lee, Y.-H. (2021) · Biosensors · doi:10.3390/bios11110411 · Cited by 40
Acceleration of Publish/Subscribe Messaging in ROS-compliant FPGA Component
Sugata, Y., Ohkawa, T., Ootsu, K., & Yokota, T. (2017) · International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies (HEART) · doi:10.1145/3120895.3120904 · Cited by 40
“Acceleration of application, such as preprocessing of SLAM [2][3] and hardwired neural network [4], has been researched actively.”
MethodEzzat, M. A., Abd El Ghany, M. A., Almotairi, S., & Salem, M. A.-M. (2021) · Sensors · doi:10.3390/s21093222 · Cited by 39
“53 s Faster RCNN (ZF) [40] FPGA Zynq (zc706) 0.”
BackgroundMethodInfluentialSPADE: A Flexible and Scalable Accelerator for SpMM and SDDMM
Gerogiannis, G., Yeşil, Ş., Lenadora, D., Cao, D., Mendis, C., & Torrellas, J. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589054 · Cited by 39
Hong, H., Choi, D., Kim, N.-J., Lee, H., Kang, B., Kang, H., & Kim, H. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01442-8 · Cited by 39
“Zhang et al. [76] applied various techniques such as tiling, double buffering, and unrolling to minimize memory access and computation costs but did not achieve high throughput.”
InfluentialTandem Processor: Grappling with Emerging Operators in Neural Networks
Ghodrati, S., Kinzer, S., Xu, H., Mahapatra, R., Kim, Y., Ahn, B. H., Wang, D. K., Karthikeyan, L., Yazdanbakhsh, A., Park, J., et al. (2024) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3620665.3640365 · Cited by 40
“In addition, on-chip address calculations require excessive number of arithmetic instructions.”
BackgroundA Survey on Hardware Accelerator Design of Deep Learning for Edge Devices
Samanta, A., Hatai, I., & Mal, A. K. (2024) · Wireless Personal Communications · doi:10.1007/s11277-024-11443-2 · Cited by 40
“Using various optimization techniques like transformation and loop tiling, Zang Chen et al. [45] quantifies its compute throughput and needed memory bandwidth.”
Han, X., Zhou, D., Wang, S., & Kimura, S. (2016) · IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD) · doi:10.1109/iccd.2016.7753296 · Cited by 39
“In [19], Vinod Nair, et al stated that ReLU has better generalized ability to prevent training model from saturating.”
BackgroundJust-In-Time Compilation for Verilog: A New Technique for Improving the FPGA Programming Experience
Schkufza, E., Wei, M., & Rossbach, C. J. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304010 · Cited by 39
“Reprogrammable hardware (FPGAs) offer compelling acceleration opportunities for high-performance applications across a wide variety of domains [19, 20, 27, 41, 42, 44, 46, 57, 60, 66, 82, 85, 90].”
BackgroundSafe Overclocking for CNN Accelerators Through Algorithm-Level Error Detection
Marty, T., Yuki, T., & Derrien, S. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.2981056 · Cited by 39
“2981056 to achieve high performance and/or to reach better energy efficiency [6], [7].”
Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence
Yang, L., Jiang, W., Liu, W., Sha, E. H. ‐., Shi, Y., & Hu, J. (2020) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045595 · Cited by 39
“Accelerator design on each PE follows [18].”
MethodA Learning Framework for n-Bit Quantized Neural Networks Toward FPGAs
Chen, J., Liu, L., Liu, Y., & Zeng, X. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980041 · Cited by 39
“Compared to prior works [30], [53], we improve the average CNN performance to 957.”
MethodEl-Maksoud, A. J. A., Ebbed, M. R., Khalil, A. H., & Mostafa, H. (2021) · IEEE Access · doi:10.1109/access.2021.3126838 · Cited by 39
“It enables building and verifying the hardware by giving better control over the architecture [25]–[27].”
BackgroundA Design Space Exploration Framework for Convolutional Neural Networks Implemented on Edge Devices
Tsimpourlas, F., Papadopoulos, L., Bartsokas, A., & Soudris, D. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2857280 · Cited by 39
“Indeed, there is a lot of work in the literature that focuses on DSE for optimizing FPGA-based and ASIC accelerators for CNNs [22]–[24].”
BackgroundFPGA-Based Convolutional Neural Network Architecture with Reduced Parameter Requirements
Hailesellasie, M., Hasan, S. R., Khalid, F., Wad, F. A., & Shafique, M. (2018) · 2018 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2018.8351283 · Cited by 38
Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW Implementation
Wess, M., Dinakarrao, S. M. P., & Jantsch, A. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2857080 · Cited by 38
“The two biggest challenges are the large size of the networks and the total number of necessary operations in feed-forward computation, since a hardware accelerator design can be bound either by the limit of parallel operations, or by the memory interface transmission rate [8], [9].”
BackgroundCHARM 2.0: Composing Heterogeneous Accelerators for Deep Learning on Versal ACAP Architecture
Zhuang, J., Lau, J., Ye, H., Yang, Z., Ji, S., Lo, J. L., Denolf, K., Neuendorffer, S., Jones, A. K., Hu, J., et al. (2024) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3686163 · Cited by 38
Self-citationDynamic Bit-width Reconfiguration for Energy-Efficient Deep Learning Hardware
Pagliari, D. J., Macii, E., & Poncino, M. (2018) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3218603.3218611 · Cited by 39
“Several works have proposed implementations of low-power HW accelerators for CNNs on FPGAs [20] and ASICs [4, 7, 14, 16, 21].”
BackgroundAn Efficient Mapping Approach to Large-Scale DNNs on Multi-FPGA Architectures
Zhang, W., Zhang, J., Shen, M., Luo, G., & Xiao, N. (2019) · Design, Automation, and Test in Europe · doi:10.23919/date.2019.8715174 · Cited by 38
“For single layer, E(i, i, x) can be solved by adapting a relatively naive method [4].”
MethodPIMBALL: Binary Neural Networks in Spintronic Memory
Resch, S., Khatamifard, S. K., Chowdhury, Z. I., Zabihi, M., Zhao, Z., Wang, J., Sapatnekar, S. S., & Karpuzcu, U. R. (2019) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3357250 · Cited by 38
“Pure floating-point heavy designs capable of handling main neural network operations [5, 47] usually impose the same data representation on the entire network, while advanced training techniques can reveal the minimum bit-width for different layers [17, 20].”
BackgroundD2NN: a fine-grained dual modular redundancy framework for deep neural networks
Li, Y., Liu, Y., Li, M., Tian, Y., Luo, B., & Xu, Q. (2019) · Annual Computer Security Applications Conference · doi:10.1145/3359789.3359831 · Cited by 38
“In real DNN deployments, the weights and activations of models are usually fixed-point to achieve high time and space efficiency [21, 32].”
BackgroundAdvantages and Limitations of Fully on-Chip CNN FPGA-Based Hardware Accelerator
Dinelli, G., Meoni, G., Rapuano, E., & Fanucci, L. (2020) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas45731.2020.9180867 · Cited by 38
A Heterogeneous Hardware Accelerator for Image Classification in Embedded Systems
Pérez, I. C., & Figueroa, M. E. (2021) · Sensors · doi:10.3390/s21082637 · Cited by 38
“Complexity-Reduction Techniques Following the experience published in the literature [36,43,46,56], we applied different techniques to MobileNet V2 to reduce its complexity before designing the architecture of the hardware accelerator.”
MethodLin, W., Adetomi, A., & Arslan, T. (2021) · Electronics · doi:10.3390/electronics10172048 · Cited by 39
“On the other hand, Reference [20] is a completed system on an FPGA board and does not release its size on a single chip, so its evaluation value E is hard to measure.”
BackgroundMethodInfluentialZhang, Y., Sun, B., Jiang, W., Ha, Y., Hu, M., & Zhao, W. (2022) · International Conference on Computer Aided Design · doi:10.1145/3508352.3549439 · Cited by 39
Stream Processing Dual-Track CGRA for Object Inference
Fan, X., Wu, D., Cao, W., Luk, W. W., & Wang, L. (2018) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2797600 · Cited by 38
“It is worth noting that only the results of convolutional layers are reported in [9] and [44].”
BackgroundMethodPipeCNN: An OpenCL-Based FPGA Accelerator for Large-Scale Convolution Neuron NetworksPreprint
Wang, D., An, J., & Xu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02450 · Cited by 38
“In [3], an accelerator design was implemented by using the Vivado-HLS tool on a Xilinx VC707 FPGA.”
MethodSC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic ComputingPreprint
Ren, A., Li, J., Li, Z., Ding, C., Qian, X., Qiu, Q., Yuan, B., & Wang, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05939 · Cited by 38
“FPGA-based accelerators, benefited from the advantages of being programmable, high degree of parallelism and short develop round, is another promising path towards the hardware implementation of DCNNs [9, 10].”
BackgroundA Ubiquitous Machine Learning Accelerator with Automatic Parallelization on FPGA
Wang, C., Gong, L., Li, X., & Zhou, X. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2990924 · Cited by 37
“Specialized accelerators have been implemented for machine learning [2], genome sequencing [25], k-means [26], neural networks [27], [28], and graph processing [8].”
MethodPhotonic diffractive generators through sampling noises from scattering media
Zhan, Z., Wang, H., Liu, Q., & Fu, X. (2024) · Nature Communications · doi:10.1038/s41467-024-55058-4 · Cited by 37
“Inspired by thepipelineprogramming schemewidely used in fi eld programmable gate arrays (FPGAs) 43 and graphics processing units (GPUs) 44 , wecandivide the total computing processinto two modules: optoelectronic operation and host computer operation.”
A high-throughput reconfigurable processing array for neural networks
Wu, E., Zhang, X., Berman, D., & Cho, I. (2017) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.23919/fpl.2017.8056794 · Cited by 37
“To maximize dot product throughput, we use the hardened summing cascade within a DSP column, as opposed to a summing tree [7][11].”
MethodDesign and Implementation of a Low-Power, Embedded CNN Accelerator on a Low-end FPGA
Khabbazan, B., & Mirzakuchaki, S. (2019) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd.2019.00102 · Cited by 37
Tang, Y., Zhang, X., Zhou, P., & Hu, J. (2022) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3505633 · Cited by 37
“[16] exploited various optimization techniques including loop unrolling, loop tiling, and loop transformation on the FPGA accelerator, and proposed a roofline model to quantitatively analyze its computing throughput and required memory bandwidth.”
BackgroundMethodInfluentialZhang, H., Liu, J., Bai, J., Li, S., Luo, L., Wei, S., Wu, J., & Kang, W. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3199440 · Cited by 37
“To efficiently support the NN models, a variety of custom AI accelerators have been proposed to meet the computing demand [6], [7], [8] , [9], [10], [11], [12].”
BackgroundA Hardware-Friendly High-Precision CNN Pruning Method and Its FPGA Implementation
Sui, X., Lv, Q., Zhi, L., Zhu, B., Yang, Y., Zhang, Y., & Tan, Z. (2023) · Sensors · doi:10.3390/s23020824 · Cited by 37
“…the two designed convolutional computation modules (Figures 6 and 9) with two conventional convolutional computation modules on FPGA: Module 1 [49], which used on-chip DSPs to implement multiplication operations traditionally, and Module 2 [50], which used on-chip LUT to implement…”
MethodScaling the Cascades: Interconnect-Aware FPGA Implementation of Machine Learning Problems
Samajdar, A., Garg, T., Krishna, T., & Kapre, N. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00061 · Cited by 37
FPGA Implementation of Convolutional Neural Networks with Fixed-Point Calculations.Preprint
Solovyev, R. A., Kalinin, A. A., Kustov, A. G., Telpukhov, D. V., & Сергеевич, Р. В. (2018) · arXiv (Cornell University) · Cited by 37
Guo, R., Chen, X. L., Wang, L., Wang, Y., Sun, H., Wei, J., Han, H., Liu, L., Wei, S., Hu, Y., et al. (2024) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2024.3402174 · Cited by 36
“X and W need to be stored and computed by the tile strategy [35].”
MethodDeep Versus Wide Convolutional Neural Networks for Object Recognition on Neuromorphic System
Alom, M. Z., Josue, T., Rahman, N., Mitchell, W., Yakopcic, C., & Taha, T. M. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489635 · Cited by 36
Optimization for Efficient Hardware Implementation of CNN on FPGA
Farrukh, F. U. D., Xie, T., Zhang, C., & Wang, Z. (2018) · IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA) · doi:10.1109/cicta.2018.8706067 · Cited by 36
“There could be 90% performance difference between two different approaches of implementation considering the same logic utilization on FPGA [1].”
BackgroundSelf-citationA Survey on Performance Optimization of High-Level Synthesis Tools
Huang, L., Li, D., Wang, K., Gao, T., & Tavares, A. (2020) · Journal of Computer Science and Technology · doi:10.1007/s11390-020-9414-8 · Cited by 36
“Most of the HLS tools provide directives for the users to manually direct the optimizing process, which makes the optimization process time-consuming and highly depend on the hardware experience of the designers.”
BackgroundMethodA RISC-V based hardware accelerator designed for Yolo object detection system
Zhang, G., Zhao, K., Wu, B., Sun, Y., Sun, L., & Liang, F. (2019) · IEEE International Conference of Intelligent Applied Systems on Engineering (ICIASE) · doi:10.1109/iciase45644.2019.9074051 · Cited by 36
“TABLE II COMPARSION WITH OTHER WORKS Platform Power (W) Performance (GOP/s) Energy Efficiency (GOPS/W) Ref.[8] Virtex-7 VXT458t 18.”
BackgroundChallenges in Energy-Efficient Deep Neural Network Training with FPGA
Tao, Y., Ma, R., Shyu, M., & Chen, S. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00208 · Cited by 36
“For example, a CNN accelerator design with uniform loop unroll factors across different convolutional layers was proposed, which achieves a performance of 61.62 GFLOPS [51].”
MethodInfluentialSamajdar, A., Qin, E., Pellauer, M., & Krishna, T. (2022) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3489517.3530506 · Cited by 36
“[17] FM ✓ ✓ Eyeriss [3] RM ✓ Alwani et al.”
In-Hardware Training Chip Based on CMOS Invertible Logic for Machine Learning
Onizawa, N., Smithson, S. C., Meyer, B. H., Gross, W. J., & Hanyu, T. (2019) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2019.2960383 · Cited by 36
“…high recognition accuracies with low-precision fixed-point or even binary-precision weight values [7], [8], inference engines have been implemented in hardware, such as field-programmable gate arrays (FPGA) and application specific integrated circuits (ASIC) [9]–[11] with high energy-efficiency.”
BackgroundOn How to Efficiently Implement Deep Learning Algorithms on PYNQ Platform
Stornaiuolo, L., Santambrogio, M. D., & Sciuto, D. (2018) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2018.00112 · Cited by 35
An Approach of Feed-Forward Neural Network Throughput-Optimized Implementation in FPGA
Novickis, R., Justs, D. J., Ozols, K., & Greitāns, M. (2020) · Electronics · doi:10.3390/electronics9122193 · Cited by 35
Optimizing Neural Networks for Efficient FPGA Implementation: A Survey
Ayachi, R., Said, Y., & Abdelali, A. B. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09530-9 · Cited by 35
“Zhang et al. [56] proposed to unroll the loops of the input channel and the output channel then the optimal unroll parameter is chosen based on the design space exploration.”
MethodFreire, P. J., Srivallapanondh, S., Anderson, M., Spinnler, B., Bex, T., Eriksson, T. A., Napoli, A., Schairer, W., Costa, N. M. S., Blott, M., et al. (2023) · Journal of Lightwave Technology · doi:10.1109/jlt.2023.3272011 · Cited by 35
“as FPGA-based accelerators have been increasingly attracting interest due to their high performance, energy efficiency, fast development cycle, and reconfiguration capability [24].”
BackgroundFFConv: An FPGA-based Accelerator for Fast Convolution Layers in Convolutional Neural Networks
Ahmad, A., & Pasha, M. A. (2020) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3380548 · Cited by 37
“Some previous works have also attempted full-precision floating-point implementations that resulted in significantly degraded performance [40].”
BackgroundMethodInfluentialAn SoC-FPGA-Based Iterative-Closest-Point Accelerator Enabling Faster Picking Robots
Kosuge, A., Yamamoto, K., Akamine, Y., & Oshima, T. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.2978722 · Cited by 35
“In addition, owing to recent intensive research activities, such as the development of quantization techniques [4], [5] and energy-efficient hardware (HW) techniques for low-power applications [6]–[8], the CNN incurs little computation time even on low-power HW.”
BackgroundA dedicated hardware accelerator for real-time acceleration of YOLOv2
Xu, K., Wang, X., Liu, X., Cao, C., Li, H., Peng, H., & Wang, D. (2020) · Journal of Real-Time Image Processing · doi:10.1007/s11554-020-00977-w · Cited by 35
“In [15], the Fig.”
BackgroundMethodInfluentialFlare: An FPGA-Based Full Precision Low Power CNN Accelerator with Reconfigurable Structure
Xu, Y., Luo, J., & Sun, W. (2024) · Sensors · doi:10.3390/s24072239 · Cited by 35
“Zhang et al. applied these methodologies to direct the Verilog HDL code generation process using HLS tools, supplemented by a DSE model to pinpoint the optimal optimization parameters required to attain their performance objectives [6].”
BackgroundMethodInfluentialNa, T., & Mukhopadhyay, S. (2016) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/2934583.2934625 · Cited by 34
“Dedicated hardware accelerator using FPGA [4] has also been proposed to improve computing throughput.”
MethodCompiler-driven FPGA virtualization with SYNERGY
Landgraf, J., Yang, T., Lin, W. Y., Rossbach, C. J., & Schkufza, E. (2021) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3445814.3446755 · Cited by 34
“FPGAs have become a compelling acceleration alternative for machine learning [16, 80, 86, 100], databases [12, 43, 61], finance [39, 58], graph processing [20, 69], communication [10, 38, 41, 76, 90], and image processing [68].”
BackgroundExploring HW/SW Co-Design for Video Analysis on CPU-FPGA Heterogeneous Systems
Zhang, X., Ma, Y., Xiong, J., Hwu, W., Kindratenko, V., & Chen, D. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3093398 · Cited by 34
Data multiplexed and hardware reused architecture for deep neural network accelerator
Raut, G., Biasizzo, A., Dhakad, N. S., Gupta, N., Papa, G., & Vishvakarma, S. K. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.018 · Cited by 34
Zeng, S., Dai, G., Zhang, N., Yang, X., Zhang, H., Zhu, Z., Yang, H., & Wang, Y. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3214113 · Cited by 34
Sang, X., Ruan, T., Li, C., Li, H., Yang, R., & Liu, Z. (2023) · Journal of Real-Time Image Processing · doi:10.1007/s11554-023-01378-5 · Cited by 34
“…of computer vision, image classification tasks have been widely used in various fields, such as lesion classification in the medical field [1], crop classification in the field of agriculture [2], pedestrian re-identification in the traffic field [3], and facial recognition in the…”
BackgroundMethodSelf-Supervised Quantization of Pre-Trained Neural Networks for Multiplierless Acceleration
Vogel, S., Springer, J., Guntoro, A., & Ascheid, G. (2019) · Design, Automation, and Test in Europe · doi:10.23919/date.2019.8714901 · Cited by 36
“A variety of dedicated hardware accelerators has been presented [2]–[4], where a majority focuses on architectural improvements rather than on optimization of the mathematical processing elements (PEs).”
BackgroundAn OpenCL(TM) Deep Learning Accelerator on Arria 10Preprint
Aydonat, U., O’Connell, S., Capalija, D., Ling, A. C., & Chiu, G. R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03534 · Cited by 34
“They empirically show that this has approximately Conversely, the work in [21, 20] use a roofline model that allows users to maximize compute resources on the FPGA given the memory bandwidth constraints.”
BackgroundMethodInfluentialA Fast and Scalable Architecture to Run Convolutional Neural Networks in Low Density FPGAs
Véstias, M. P., Duarte, R. P., de Sousa, J. T., & Neto, H. C. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.103136 · Cited by 33
Normalization and dropout for stochastic computing-based deep convolutional neural networks
Li, J., Yuan, Z., Li, Z., Ren, A., Ding, C., Draper, J. T., Nazarian, S., Qiu, Q., Yuan, B., & Wang, Y. (2017) · Integration · doi:10.1016/j.vlsi.2017.11.002 · Cited by 33
“A hardware/software co-designed library called Caffeine was proposed by Zhang et al. to efficiently accelerate the DCNN on FPGAs with integration into the industry-standard software deep learning framework Caffe [26].”
MethodInfluentialAlgorithm-hardware co-design of single shot detector for fast object detection on FPGAs
Ma, Y., Zheng, T., Cao, Y. K., Vrudhula, S. B. K., & Seo, J.-S. (2018) · doi:10.1145/3240765.3240775 · Cited by 33
“With 8,192 MAC units operated at 300 MHz, the theoretical maximum throughput of Stratix 10 is 4,915 GOPS, which is 3.3× larger than the Arria 10 maximum throughput of 1,474 GOPS.”
MethodInfluentialInS-DLA: An In-SSD Deep Learning Accelerator for Near-Data Processing
Liang, S., Wang, Y., Liu, C., Li, H., & Li, X. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00035 · Cited by 33
“Although the performance of InS-DLA is lower than that in [40], InS-DLA consumes much fewer hardware resources and power.”
BackgroundOptimizing Performance of Convolutional Neural Network Using Computing Technique
Samudre, P., Shende, P., & Jaiswal, V. K. (2019) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct45611.2019.9033876 · Cited by 33
“As per research, the implemented CNN accelerator on a VC707 FPGA board and compared it to earlier ways [8].”
MethodMalviya, V., Mukherjee, I., & Tallur, S. G. (2022) · IEEE Sensors Letters · doi:10.1109/lsens.2022.3159972 · Cited by 33
Giamougiannis, G., Tsakyridis, A., Moralis‐Pegios, M., Totovic, A. R., Kirtas, M., Passalis, N., Tefas, A., Lazovsky, D., & Pleros, N. (2022) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2022.3228318 · Cited by 33
“…of highly parallelized hardware capable to undertake and accelerate these operations, with several graphic processing units (GPUs) [1], field programmable gate arrays [2] and application specific integrated circuits (ASICs) [3] having been demonstrated and fabricated within the last decade.”
BackgroundDielectric Elastomer-Based Actuators: A Modeling and Control Review for Non-Experts
Medina, H., Farmer, C., & Liu, I. (2024) · Actuators · doi:10.3390/act13040151 · Cited by 33
A memory-based realization of a binarized deep convolutional neural network
Nakahara, H., Yonekawa, H., Sasao, T., Iwamoto, H., & Motomura, M. (2016) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2016.7929552 · Cited by 33
“Year 2015 [15] 2016 [14] Proposed Platform Virtex7 Zynq Virtex7 VX485T XC7Z045 XC7V690T Clock (MHz) 100 150 450 Memory Bandwidth (GB/s) 12.”
Perspective: Uniform switching of artificial synapses for large-scale neuromorphic arrays
Tan, S. H., Lin, P., Yeon, H., Choi, S., Park, Y., & Kim, J. (2018) · APL Materials · doi:10.1063/1.5049137 · Cited by 33
Compute-Efficient Neural-Network Acceleration
Wu, E., Zhang, X., Berman, D., Cho, I., & Thendean, J. (2019) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3289602.3293925 · Cited by 33
ReNEW: Enhancing Lifetime for ReRAM Crossbar Based Neural Network Accelerators
Wen, W., Zhang, Y., & Yang, J. (2019) · IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD) · doi:10.1109/iccd46524.2019.00074 · Cited by 32
“Conventional CMOS-based general purpose processor such as multi-core CPU [5] and GPGPU [1], or specialized hardware accelerators, such as FPGA [6] and ASIC designs [7], [8], are intensively studied and proposed with software and hardware optimizations for neural network applications, however, they still suffer from th…”
MethodA complete photonic integrated neuron for nonlinear all-optical computing
Yan, T., Guo, Y., Zhou, T., Shao, G., Li, S., Huang, R., Dai, Q., & Fang, L. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00866-x · Cited by 32
“…limits, existing electronic computing hardware such as central processing units 7 , graphics processing units 2 , field-programmable gate arrays 8 and application-specific integrated circuits 9 faces substantial computational burdens, suffering a saturation of computing speeds and energy…”
Implementation of Deep Learning-Based Automatic Modulation Classifier on FPGA SDR Platform
Tang, Z., Li, S., & Lijuan, Y. (2018) · Electronics · doi:10.3390/electronics7070122 · Cited by 32
“…resources in FPGA devices. circulation block and transform), computing throughput, and the required memory bandwidth so as to find the best performance and the lowest FPGA resource Expanding along different loop dimensions generates different implementation variables. demand solutions [26].”
BackgroundMethodInfluentialCost-effective stochastic MAC circuits for deep neural networks
Sim, H., & Lee, J. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.04.017 · Cited by 32
NeuronLink: An Efficient Chip-to-Chip Interconnect for Large-Scale Neural Network Accelerators
Xiao, S., Guo, Y., Liao, W., Deng, H., Luo, Y., Zheng, H., Wang, J., Li, C., Li, G., & Yu, Z. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2020.3008185 · Cited by 32
“This has motivated various computing platforms [4]–[7], like CPU, GPU, ASIC, and field-programmable gate array (FPGA), to accelerate DNN operations.”
MethodJiang, S., Ma, Z., Zeng, X., Xu, C., Zhang, M., Zhang, C., & Liu, Y. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155435 · Cited by 33
“…its massively parallel computing units and on-chip storage banks, which saves a large portion of overheads in general-purpose processors, like instruction fetch, cache miss, task scheduling etc. Existing FPGA accelerators achieved 3-30x speedup compared with CPU on different DNN models [17], [15].”
BackgroundSelf-citationLight-Weight CNN Enabled Edge-Based Framework for Machine Health Diagnosis
Mukherjee, I., & Tallur, S. G. (2021) · IEEE Access · doi:10.1109/access.2021.3088237 · Cited by 32
“Zhang et al. present various strategies to realize fast, parallel processing designs to develop CNN accelerators but mention the challenge of encountering mismatch of computation throughput with the memory bandwidth provided by an FPGA platform [17].”
BackgroundFPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive ReviewPreprint
Jiang, J., Zhou, Y., Gong, Y., Yuan, H., & Liu, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.13461 · Cited by 32
“Unlike P k and P v , which are constrained by kernel size and feature characteristics, the combined approach of P c and P f offers greater flexibility [6], [28], [75]–[78].”
Ghaffari, A., & Savaria, Y. (2020) · Electronics · doi:10.3390/electronics9122200 · Cited by 31
“CNN2Gate is faster than [21,22] in terms of latency and throughput.”
BackgroundMa, X., Zhang, Y., Yuan, G., Ren, A., Li, Z., Han, J., Hu, J., & Wang, Y. (2018) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed.2018.8357306 · Cited by 31
A Data-Center FPGA Acceleration Platform for Convolutional Neural Networks
Yu, X., Gao, J., Wang, Y., Miao, J., Wu, E., Zhang, H., Meng, Y., Zhang, B., Min, B., & Chen, D. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00032 · Cited by 31
DUET: Boosting Deep Neural Network Efficiency on Dual-Module Architecture
Liu, L., Qu, Z., Deng, L., Tu, F., Li, S., Hu, X., Gu, Z., Ding, Y., & Xie, Y. (2020) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro50266.2020.00066 · Cited by 31
“Academia and industry have proposed various architectures for the acceleration of DNNs [8], [10], [11], [12], [15], [17], [18], [28], [29], [31], [35], [43], [49], and here we focus on algorithm-architecture co-design to save computations and data accesses.”
BackgroundA Self-Test Framework for Detecting Fault-induced Accuracy Drop in Neural Network Accelerators
Meng, F., Hosseini, F. S., & Yang, C. (2021) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3394885.3431519 · Cited by 31
“To accommodate DNN applications on resource-constrained edge devices, many DNN accelerators have been developed, including both traditional ASIC and FPGA-based accelerators [23] as well as emerging processingin-memory (PIM) accelerators [18].”
MethodZokaee, F., & Jiang, L. (2021) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3466752.3480041 · Cited by 31
“A convolutional layer is a 6-nested loop [59] belonging to a basic block.”
BackgroundSGCNAX: A Scalable Graph Convolutional Neural Network Accelerator with Workload Balancing
Li, J., Zheng, H., Wang, K., & Louri, A. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3133691 · Cited by 31
“Specifically, if the innermost loop is irrelevant to a matrix, i.e., the loop iterator does not appear in the access function of the matrix [27], there will be redundant memory operations between different loop iterations which can be eliminated to reduce memory access operations.”
BackgroundLi, N., Takaki, S., Yoichi, T., & Kitazawa, H. (2016) · IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) · doi:10.1109/ssiai.2016.7459201 · Cited by 31
Beyond the Roofline: Cache-Aware Power and Energy-Efficiency Modeling for Multi-Cores
Ilić, A., Pratas, F., & Sousa, L. A. (2016) · IEEE Transactions on Computers · doi:10.1109/tc.2016.2582151 · Cited by 31
“The ORM was also applied to aid hardware/software co-design [2], [18] and for different device architectures [19].”
MethodA High-Performance Accelerator for Large-Scale Convolutional Neural Networks
Sun, F., Wang, C., Gong, L., Xu, C., Zhang, Y., Lu, Y., Li, X., & Zhou, X. (2017) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa/iucc.2017.00099 · Cited by 31
“To illustrate this problem, here we take state-of-the-art design [7] as an example.”
BackgroundMethodInfluentialReconfigurable Convolutional Kernels for Neural Networks on FPGAs
Hardieck, M., Kumm, M., Möller, K., & Zipf, P. (2019) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3289602.3293905 · Cited by 31
“Most of the previous FPGA-based CNN implementations [1, 3, 11, 14, 21, 23, 24, 27, 31, 34], implement the required sum-of-products (SOP) operation in a parallel fashion.”
BackgroundMethod
Metadata
- Authors
- Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J.
- Affiliations
- Peking University (CN)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
- Publisher
- ACM
- Published
- 2015-02-22
- Pages
- 161-170
- OpenAlex
- W2094756095
- Cited by
- 2,158
- References
- 17
- Citations by year
- 2026: 78 · 2025: 116 · 2024: 145 · 2023: 162 · 2022: 233 · 2021: 253 · 2020: 258 · 2019: 303 · 2018: 281 · 2017: 221 · 2016: 90 · 2015: 15
- FWCI
- 133.92
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) (publishedVersion)
- Funders
- C-FAR
- NSF China (61202072)
- National High Technology Research and Development Program of China (2012AA010902)
- RFDP (20110001110099)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
- Keywords
- FPGA accelerator, deep convolutional neural networks, roofline model, memory bandwidth, loop tiling, loop transformation, FPGA resource utilization, image recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.