Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Convolutional neural network (CNN) has been widely employed for image recognition because it can achieve high accuracy by emulating behavior of optic nerves in living creatures. Recently, rapid growth of modern applications based on deep learning algorithms has further improved research and implementations. Especially, various accelerators for deep CNN have been proposed based on FPGA platform because it has advantages of high performance, reconfigurability, and fast development round, etc. Although current FPGA accelerators have demonstrated better performance over generic processors, the accelerator design space has not been well exploited. One critical problem is that the computation throughput may not well match the memory bandwidth provided an FPGA platform. Consequently, existing approaches cannot achieve best performance due to under-utilization of either logic resource or memory bandwidth. At the same time, the increasing complexity and scalability of deep learning applications aggravate this problem. In order to overcome this problem, we propose an analytical design scheme using the roofline model. For any solution of a CNN design, we quantitatively analyze its computing throughput and required memory bandwidth using various optimization techniques, such as loop tiling and transformation. Then, with the help of rooine model, we can identify the solution with best performance and lowest FPGA resource requirement. As a case study, we implement a CNN accelerator on a VC707 FPGA board and compare it to previous approaches. Our implementation achieves a peak performance of 61.62 GFLOPS under 100MHz working frequency, which outperform previous approaches significantly.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another option is to use field programable gate arrays (FPGAs), which consume much less energy than GPUs while providing an intermediate computing efficiency between CPUs and GPUs13–17.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [15]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“Figure 2 shows a real-life CNN application, taken from [9].”
BackgroundMethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“The algorithm receives three 224x224 input images that are from an original 256x256 three-channel RGB image.”
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992 · cited 1×
“Convolutional neural network (CNN), a well-known deep learning architecture extended from artificial neural network, has been extensively adopted in various applications, which include video surveillance, mobile robot vision, image search engine in data centers, etc [6] [7] [8] [10] [14].”
BackgroundMinimizing Computation in Convolutional Neural Networks
Cong, J., & Xiao, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11179-7_36 · Cited by 372 · cited 1×
“In the feedforward computation perspective, a previous study [5] proved that convolution operations will occupy over 90% of the computation time.”
BackgroundSelf-citationCNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343 · cited 8×
“Among these approaches, FPGA based accelerators have attracted more and more attention of researchers because they have advantages of good performance, high energy efficiency, fast development round, and capability of reconfiguration [1] [2] [3] [6] [12] [14].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- D. Aysegul , J. Jonghoon , G. Vinayak , K. Bharadwaj , C. Alfredo , M. Berin , and C. Eugenio . Accelerating deep neural networks on mobile processor with embedded programmable logic . In NIPS 2013 . IEEE, 2013 . D. Aysegul, J. Jonghoon, G. Vinayak, K. Bharadwaj, C. Alfredo, M. Berin, and C. Eugenio. Accelerating deep neural networks on mobile processor with embedded programmable logic. In NIPS 2013. IEEE, 2013.
Cited by
Items 301–400 of 1,002
Li, S., Luo, Y., Sun, K., Yadav, N., & Choi, K. K. (2020) · IEEE Access · doi:10.1109/access.2020.3000009 · Cited by 75
“62 GFLOPS in the work of Zhang et al. [20] and 84 .”
MethodDeepX: Deep Learning Accelerator for Restricted Boltzmann Machine Artificial Neural Networks
Kim, L.-W. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2665555 · Cited by 75
“Reference [17] presented an automated method to explore the design space to generate optimized”
BackgroundMethodOn-Chip Memory Technology Design Space Explorations for Mobile Deep Neural Network Accelerators
Li, H., Bhargava, M., Whatmough, P. N., & Wong, H. ‐. P. (2019) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3316781.3317874 · Cited by 75
“Finally, DSE for DNN accelerators has been considered before [19, 20].”
MethodLUTNet: Learning FPGA Configurations for Highly Efficient Neural Network Inference
Wang, E., Davis, J. J., Cheung, P. Y. K., & Constantinides, G. A. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2978817 · Cited by 75
“Zhang et al. proposed tiling along both input and output channels [24].”
BackgroundAn Energy-Efficient Digital ReRAM-Crossbar-Based CNN With Bitwise Parallelism
Ni, L., Liu, Z., Yu, H., & Joshi, R. V. (2017) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2017.2697910 · Cited by 74
“Moreover, compared to FPGA-based and CMOSASIC-based conventional CNN, the digital-ReRAM-based implementation is 4155 times and 62 times smaller power.”
BackgroundInfluentialA Dynamic ReLU on Neural Network
Si, J., Harris, S. L., & Yfantis, E. A. (2018) · IEEE Dallas Circuits and Systems Conference (DCAS) · doi:10.1109/dcas.2018.8620116 · Cited by 74
“Systems built on an FPGA platform, however, offer algorithm-specific hardware and low power consumption [6][7], are easy to pipeline architecturally [6][8] and algorithmically [9], and allow for quantization and compression of weights [7][10].”
BackgroundA multichannel optical computing architecture for advanced machine vision
Xu, Z., Yuan, X., Zhou, T., & Fang, L. (2022) · Light Science & Applications · doi:10.1038/s41377-022-00945-y · Cited by 74
“Despite the rapid development of neural processing units (NPUs) in recent years 12 – 15 , the performance and energy ef fi ciency of conventional silicon-based computing devices are restricted by the pla-teauing of Moore ’ s law 16 , leading to the limited scaling of electronic transistors in…”
BackgroundOpenFPGA: An Opensource Framework Enabling Rapid Prototyping of Customizable FPGAs
Tang, X., Giacomin, E., Alacchi, A., Chauviere, B., & Gaillardon, P. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00065 · Cited by 74
On Neural Architecture Search for Resource-Constrained Hardware PlatformsPreprint
Lu, Q. H., Jiang, W., Xu, X., Shi, Y., & Hu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00105 · Cited by 74
“For the single-layer accelerator design, we adopted the widely used tile-based paradigm [24].”
MethodNEBULA: A Neuromorphic Spin-Based Ultra-Low Power Architecture for SNNs and ANNs
Singh, S., Sarma, A., Jao, N., Pattnaik, A., Lu, S. K., Yang, K., Sengupta, A., Narayanan, V., & Das, C. R. (2020) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca45697.2020.00039 · Cited by 73
“Despite several hardware advancements with accelerators like GPUs [49], [60], [84], TPUs [39] and other custom ASICs [3], [53], [65], [67], [91], present day computing system is far from achieving the unparalleled power efficiency of a ∼ 20W human brain [19], [86].”
BackgroundInP photonic integrated multi-layer neural networks: Architecture and performance analysis
Shi, B., Calabretta, N., & Stabile, R. (2021) · APL Photonics · doi:10.1063/5.0066350 · Cited by 73
Cong, J., Wei, P., Yu, C. H., & Zhang, P. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac.2018.8465940 · Cited by 73
Self-citationHu-Fu: Hardware and Software Collaborative Attack Framework Against Neural Networks
Li, W., Yu, J., Ning, X., Wang, P., Wei, Q., Wang, Y., & Yang, H. (2018) · 2018 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) · doi:10.1109/isvlsi.2018.00093 · Cited by 58
A CGRA-Based Approach for Accelerating Convolutional Neural Networks
Masakazu, T., Takamaeda-Yamazaki, S., Yao, J., & Nakashima, Y. (2015) · IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC) · doi:10.1109/mcsoc.2015.41 · Cited by 72
“The implemented model can be simulated on both standard CPUs and GPUs.”
MethodData and Hardware Efficient Design for Convolutional Neural Network
Lin, Y.-J., & Chang, T. (2017) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2017.2759803 · Cited by 72
“Reference [29] had taken a step further to”
BackgroundPUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning InferencePreprint
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltín, M., Williams, R. S., Faraboschi, P., Hwu, W., Strachan, J. P., Roy, K., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10351 · Cited by 73
“In the digital realm, accelerators can be classified as weight stationary spatial architectures [15, 17, 36, 43, 85, 93], output stationary spatial architectures [33, 47, 86], spatial architectures with no local reuse [19, 20, 117], and row stationary spatial architectures [21].”
BackgroundZhou, Y., Yang, M., Guo, C., Leng, J., Liang, Y. E., Chen, Q., Guo, M., & Zhu, Y. (2021) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc53511.2021.00029 · Cited by 71
Scaling Binarized Neural Networks on Reconfigurable Logic
Fraser, N. J., Umuroglu, Y., Gambardella, G., Blott, M., Leong, P. H. W., Jahre, M., & Vissers, K. (2017) · doi:10.1145/3029580.3029586 · Cited by 70
“…basic architectures: 1) a single processing engine , usually in the form of a systolic array , which processes each layer sequentially [3, 5, 19, 28]; 2) a streaming architecture [1, 26], consisting of one processing engine per network layer; 3) a vector processor [8] with instructions specific…”
Distributed Deep Convolutional Neural Networks for the Internet-of-Things
Disabato, S., Roveri, M., & Alippi, C. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3062227 · Cited by 70
“At the hardware level, the best technological performances with minimum power consumption are achieved by ad-hoc hardware computing platforms for DL as specific processors [20], or configurable FPGA solutions [21], [22].”
MethodWei, X., Liang, Y. E., & Cong, J. (2019) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3316781.3317875 · Cited by 70
Self-citationA Systematic Approach to Blocking Convolutional Neural NetworksPreprint
Yang, X., Pu, J., Rister, B., Bhagdikar, N., Richardson, S., Kvatinsky, S., Ragan‐Kelley, J., Pedram, A., & Horowitz, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04209 · Cited by 70
“Both Peemen et al. [28] and Zhang et al. [42] explored blocking in an FPGA CNN accelerator, but considered only two levels of memory and blocking, and sought only to minimize off-chip bandwidth, not total memory energy.”
BackgroundMethodInfluentialTolerating Soft Errors in Deep Learning Accelerators with Reliable On-Chip Memory Designs
Azizimazreah, A., Gu, Y., Gu, X., & Chen, L. (2018) · 2018 IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas.2018.8515692 · Cited by 56
“The parametrized DNN accelerator is implemented in RTL Verilog, whose architecture is based on the paper [35].”
MethodPerformance Modeling for CNN Inference Accelerators on FPGA
Ma, Y., Cao, Y. K., Vrudhula, S. B. K., & Seo, J.-S. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2019.2897634 · Cited by 68
“…(or ping-pong buffering) technique, the performance of the CNN accelerator will be limited by either the computation delay or the DRAM transfer delay, and the actual bound will be determined by the values of the associated design parameters, as described by the roofline model in [7] and [9].”
BackgroundMethodInfluentialTanberk, S., Kilimci, Z. H., Tükel, D. B., Uysal, M., & Akyokuş, S. (2020) · IEEE Access · doi:10.1109/access.2020.2968529 · Cited by 68
“one pixel in one output feature map [26].”
BackgroundGenc, H., Haj-Ali, A., Iyer, V., Amid, A., Mao, H., Wright, J., Schmidt, C., Zhao, J., Ou, A. J., Banister, M., et al. (2019) · arXiv (Cornell University) · Cited by 68
Naeem, M., Pietro, G. D., & Coronato, A. (2021) · Sensors · doi:10.3390/s22010309 · Cited by 67
“Many characteristics may be investigated for the implementation level, including FPGA designs [311] and advanced GPU [312].”
BackgroundNNPIM: A Processing In-Memory Architecture for Neural Network Acceleration
Gupta, S., Imani, M., Kaur, H., & Simunic, T. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2903055 · Cited by 67
“Earlier work proposed several FPGA-based and ASIC designs [6], [7], [8], [9] to accelerate neural networks.”
MethodAn Experimental Study of Reduced-Voltage Operation in Modern FPGAs for Neural Network Acceleration
Salami, B., Onural, E. B., Yüksel, İ. E., Koç, F., Ergin, O., Kestelman, A. C., Ünsal, O., Sarbazi‐Azad, H., & Mutlu, O. (2020) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn48063.2020.00032 · Cited by 67
“Efficient exploitation of the underlying hardware is fundamental for power-efficiency, using techniques like resource partitioning [105] and reuse [88, 131].”
MethodFarrukh, F. U. D., Zhang, C., Jiang, Y., Zhang, Z., Wang, Z., Wang, Z., & Jiang, H. (2020) · IEEE Open Journal of Circuits and Systems · doi:10.1109/ojcas.2020.3007334 · Cited by 67
“Only FPGA’15 [18] design is running the AlexNet model and the rest of the designs in Table 9 are running the Tiny-YOLO-v2 model on FPGA platform to test their proposed designs.”
BackgroundMethodInfluentialSelf-citationAddNet: Deep Neural Networks Using FPGA-Optimized Multipliers
Faraone, J., Kumm, M., Hardieck, M., Zipf, P., Liu, X., Boland, D., & Leong, P. H. W. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2939429 · Cited by 66
“Field-programmable gate array (FPGA) implementations have demonstrated improved latency and power efficiency compared with central processing unit (CPU) and graphics processing unit (GPU) technologies (see [2] and [3]).”
BackgroundApplication of artificial intelligence in reverse logistics: A bibliometric and network analysis
Bhowmik, O., Chowdhury, S., Ashik, J. H., Mahmud, G., Khan, M. M., & Hossain, N. U. I. (2024) · Supply Chain Analytics · doi:10.1016/j.sca.2024.100076 · Cited by 67
Neuro.ZERO: a zero-energy neural network accelerator for embedded sensing and inference systems
Lee, S., & Nirjon, S. (2019) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3356250.3360030 · Cited by 65
“DNN inference accelerators using FPGAs such as [30, 93, 112, 122] have been widely studied due to their high performance and reconfigurability.”
MethodFPGA-based accelerator for object detection: a comprehensive survey
Zeng, K., Ma, Q., Wu, J. W., Chen, Z., Shen, T., & Yan, C. C. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04415-5 · Cited by 66
“Zhang et.al proposed a roofline-based method to accelerate CNNs on FPGAs [54].”
BackgroundMethodInfluentialA Uniform Architecture Design for Accelerating 2D and 3D CNNs on FPGAs
Liu, Z., Chow, P., Xu, J., Jiang, J., Dou, Y., & Zhou, J. N. (2019) · Electronics · doi:10.3390/electronics8010065 · Cited by 65
“For example, the designs in [13,14] adopt customized computation engines to compute 2D convolutions.”
MethodBridge the Gap between Neural Networks and Neuromorphic Hardware with a Neural Network Compiler
Ji, Y., Zhang, Y., Chen, W., & Xie, Y. (2018) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3173162.3173205 · Cited by 64
“They are custom architectures [11, 12, 15, 21, 23, 24, 38, 39, 46, 47, 56, 57, 59, 61, 64, 67, 68] to accelerate mature ANN models.”
MethodA Power-Efficient Optimizing Framework FPGA Accelerator Based on Winograd for YOLO
Bao, C., Xie, T., Feng, W., Chang, L., & Yu, C. (2020) · IEEE Access · doi:10.1109/access.2020.2995330 · Cited by 66
“Zhang et al. [14] proposed a ping-pong structure for the buffer to achieve FPGA memory optimization to ensure fast data interaction, and they analysed the design space of the proposed FPGA accelerator with a roofline model to solve the problem of optimizing computing resources and bandwidth.”
MethodLayerweaver: Maximizing Resource Utilization of Neural Processing Units via Layer-Wise Scheduling
Oh, Y. H., Kim, S., Jin, Y., Son, S., Bae, J., Lee, J.-S., Park, Y., Kim, D. U., Ham, T. J., & Lee, J. W. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00056 · Cited by 65
“Note that those baselines used for our evaluation substantially outperform layer-wise double buffering [32], which is used as a baseline scheme of AI-MT.”
BackgroundMethodMachine learning on FPGAs to face the IoT revolution
Zhang, X., Ramachandran, A., Zhuge, C., He, D., Zuo, W., Cheng, Z., Rupnow, K., & Chen, D. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203862 · Cited by 65
“The work presented in [14] explores the design space of loop optimizations in a CNN implementation to locate the best implementation point.”
BackgroundA Survey on Vertical Federated Learning: From a Layered PerspectivePreprint
Liu, Y., Chai, D., Zhang, J., Jin, Y., Wang, L., Liu, H., Han, T., Xu, Q., & Chen, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.01829 · Cited by 65
“FPGA has been applied in traditional machine learning [221], cryptography [158], etc.”
MethodCho, M., & Kim, Y. (2021) · Electronics · doi:10.3390/electronics10222859 · Cited by 65
“Increasing the efficiency of FPGA resources is also a challenge in CNN accelerator design [15,21–23].”
BackgroundMethodFPGA-Based Vehicle Detection and Tracking Accelerator
Zhai, J., Li, B., Lv, S., & Zhou, Q. (2023) · Sensors · doi:10.3390/s23042208 · Cited by 64
“[29] proposed optimizing the on-chip cache using ping-pong operations to hide the data transfer latency and designed it to search the accelerator optimization space using the roofline model.”
MethodDesigning Hardware for Machine Learning: The Important Role Played by Circuit Designers
Sze, V. (2017) · IEEE Solid-State Circuits Magazine · doi:10.1109/mssc.2017.2745798 · Cited by 64
Giamougiannis, G., Tsakyridis, A., Moralis‐Pegios, M., Mourgias-Alexandris, G., Totovic, A. R., Dabos, G., Kirtas, M., Passalis, N., Tefas, A., Kalavrouziotis, D., et al. (2023) · Advanced Photonics · doi:10.1117/1.ap.5.1.016004 · Cited by 63
Advancements in Accelerating Deep Neural Network Inference on AIoT Devices: A Survey
Cheng, L., Gu, Y., Liu, Q., Yang, L., Liu, C., & Wang, Y. (2024) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2024.3353176 · Cited by 64
Energy-efficient and high-throughput FPGA-based accelerator for Convolutional Neural Networks
Feng, G., Hu, Z., Chen, S., & Wu, F. (2016) · IEEE International Conference on Solid-State and Integrated Circuit Technology · doi:10.1109/icsict.2016.7998996 · Cited by 63
“The bottlenecks of FPGA-based CNN can be eliminated by atom arithmetic level of legitimately planning operations like addition, multiplication [5][8], and advanced high-level optimization of parallelizing loops, increasing bandwidth with scheduled hardware resources [6][9].”
BackgroundMethodInfluentialA Winograd-Based Integrated Photonics Accelerator for Convolutional Neural Networks
Mehrabian, A., Miscuglio, M., Alkabani, Y., Sorger, V. J., & El‐Ghazawi, T. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2957443 · Cited by 63
“FPGA [16], [51]–[54] and GPU implementations [16].”
FPGA-Based Acceleration for Bayesian Convolutional Neural Networks
Fan, H., Ferianc, M., Que, Z., Liu, S., Niu, X., Rodrigues, M. R. D., & Luk, W. W. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3160948 · Cited by 62
FPGA‐accelerated deep convolutional neural networks for high throughput and energy efficiency
Qiao, Y., Shen, J., Xiao, T., Yang, Q., Wen, M., & Zhang, C. (2016) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.3850 · Cited by 62
“Recently, the authors of [10] discussed the design space of a CNN accelerator, but it still needs to adjust the accelerator structure to a fixed network.”
BackgroundInfluentialSelf-citationTowards Efficient Microarchitectural Design for Accelerating Unsupervised GAN-Based Deep Learning
Song, M., Zhang, J., Chen, H., & Li, T. (2018) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00016 · Cited by 62
Sim, J., Lee, S., & Kim, L. (2019) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2935251 · Cited by 62
“This reuse chance is, in fact, leveraged by some previous research studies [21], [22].”
ResultOne-Dimensional Deep Learning Architecture for Fast Fluorescence Lifetime Imaging
Xiao, D., Chen, Y., & Li, D. (2021) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2021.3049349 · Cited by 62
Programming and Synthesis for Software-defined FPGA Acceleration: Status and Future Prospects
Lai, Y., Ustun, E., Xiang, S., Fang, Z., Rong, H., & Zhang, Z. (2021) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3469660 · Cited by 63
“The field of FPGA-based acceleration is also more vibrant than ever, as there is a growing number of HLS-synthesized designs that accelerate a plethora of emerging applications such as deep learning [247, 263, 270, 277], genomics [102, 116], graph analytics [30], and packet processing [118, 244].”
BackgroundHardware Acceleration for Machine Learning
Zhao, R., Luk, W. W., Niu, X., Shi, H.-F., & Wang, H. (2017) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2017.127 · Cited by 61
“[8] use the roofline model [9] and data dependence analysis to optimise a convolution-only AlexNet-based CNN architecture.”
MethodPark, S., Park, J.-Y., Bong, K., Shin, D., Lee, J., Choi, S., & Yoo, H. (2016) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2015.2504563 · Cited by 61
“But, deep learning environments consume lots of power ( W) and has low energy-efficiency (11–12 GFLOPS/W, GPU [9]), and deep inference environments has less performance (61GFLOPS, FPGA [10] and 160GFLOPS, DSP [11]) than deep learning operation requires.”
BackgroundMixed-signal circuits for embedded machine-learning applications
Murmann, B., Bankman, D., Chai, E., Miyashita, D., & Yang, L. (2015) · 2015 49th Asilomar Conference on Signals, Systems and Computers · doi:10.1109/acssc.2015.7421361 · Cited by 51
“The study of [29]”
A Convolutional Neural Network Fully Implemented on FPGA for Embedded Platforms
Bettoni, M., Urgese, G., Kobayashi, Y., Macii, E., & Acquaviva, A. (2017) · IEEE International Conference on New Generation of Circuits and Systems (NGCAS) · doi:10.1109/ngcas.2017.16 · Cited by 61
“We configured the HW-CNN with the Alex-Net model structure and parameters used in the Cong work [9].”
MethodInfluentialScale-out acceleration for machine learning
Park, J., Sharma, H., Mahajan, D., Kim, J. K., Olds, P., & Esmaeilzadeh, H. (2017) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3123939.3123979 · Cited by 61
“There is a large body of work on single-node accelerator design for ML [3, 5–10, 28– 40, 40, 41, 41–49].”
BackgroundGilan, A. A., Emad, M., & Alizadeh, B. (2019) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2019.2922372 · Cited by 61
“In [3], a roofline mathematical model of the convolution operation is proposed.”
MethodNeuromorphic Hardware Architecture Using the Neural Engineering Framework for Pattern Recognition
Wang, R., Thakur, C. S., Cohen, G., Hamilton, T. J., Tapson, J. C., & André, V. S. (2017) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2017.2666883 · Cited by 61
“To solve this problem, scientists have been developing hardware implementations of conventional artificial neural networks over the past two decades [1]–[7].”
BackgroundDinelli, G., Meoni, G., Rapuano, E., Benelli, G., & Fanucci, L. (2019) · International Journal of Reconfigurable Computing · doi:10.1155/2019/7218758 · Cited by 60
“Themainissue about the implementation of a CNN-based model on an FPGA regards the limitation in terms of available hardware resources (combinatorial elements, sequential elements, Digital Signal Processors (DSPs), ram blocks, etc.) of such devices [11, 16, 17].”
BackgroundArjoune, Y., & Faruque, S. (2020) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc47524.2020.9031117 · Cited by 60
“At the implementation level, several characteristics can be explored such as advanced GPU [48] and FPGA designs [49].”
BackgroundNguyen, D. T., Je, H., Nguyen, T. N., Ryu, S., Lee, K., & Lee, H. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3153288 · Cited by 60
A Survey on Machine Learning Accelerators and Evolutionary Hardware Platforms
Bavikadi, S., Dhavlle, A., Ganguly, A., Haridass, A., Hendy, H., Merkel, C. E., Reddi, V. J., Sutradhar, P. R., Joseph, A., & Dinakarrao, S. M. P. (2022) · IEEE Design and Test · doi:10.1109/mdat.2022.3161126 · Cited by 60
COSMOS: Coordination of High-Level Synthesis and Memory Optimization for Hardware Accelerators
Piccolboni, L., Mantovani, P., Guglielmo, G. D., & Carloni, L. P. (2017) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3126566 · Cited by 60
“Several accelerator designs have been proposed in the literature to realize hardware implementations that execute important computational kernels more e˝ciently than corresponding software executions [9, 10, 23, 29, 41, 48]. The accelerators can be located either inside (tightly-coupled) or outside (loosely-coupled) t…”
BackgroundA High Throughput Acceleration for Hybrid Neural Networks With Efficient Resource Management on FPGA
Yin, S., Tang, S., Lin, X., Ouyang, P., Tu, F., Liu, L., & Wei, S. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2821561 · Cited by 60
“Since CLs take a most portion of computation in the hybridNN, many prior work [9]–[12], [23] focus on CLs acceleration, including the design of an efficient accelerating engine and the mapping mechanism from a large CLs to a limited accelerating engine.”
BackgroundMethodInfluentialDouble MAC on a DSP: Boosting the Performance of Convolutional Neural Networks on FPGAs
Lee, S., Kim, D. W., Nguyen, D. T., & Lee, J. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2824280 · Cited by 61
“, [2]), but recent designs use fixed-point precisions”
BackgroundMethodInfluentialSpectral-based convolutional neural network without multiple spatial-frequency domain switchings
Ayat, S. O., Khalil-Hani, M., Rahman, A. A.-H. A., & Abdellatef, H. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.094 · Cited by 61
Zhao, Y. K., Li, C., Wang, Y., Xu, P., Zhang, Y., & Lin, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053977 · Cited by 60
“For example, the first well-optimized FPGA DNN accelerator [16] uses loop tiling; the DianNao series [13, 17] is an early effort on synthesis based ASIC accelerators; Eyeriss proposes a row-stationary dataflow [14] to reduce expensive DRAM accesses; and Google TPUs [11, 12] use a systolic array to achieve high through…”
MethodRodríguez, I., Campos, C., & Fdez‐Riverola, F. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06830-w · Cited by 61
Huang, M., Liu, Y., Man, C., Li, K., Cheng, Q., Mao, W., & Yu, H. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3178474 · Cited by 60
“Generally, the sparse based strategy can greatly decrease the MAC operations, thus accelerating the CNN process and increase the throughput, but the network accuracy will be largely degraded.”
BackgroundFPGA architecture for deep learning and its application to planetary robotics
Gankidi, P., & Thangavelautham, J. (2017) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero.2017.7943929 · Cited by 59
In-Depth Analysis on Microarchitectures of Modern Heterogeneous CPU-FPGA Platforms
Choi, Y., Cong, J., Fang, Z., Hao, Y., Reinman, G., & Wei, P. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3294054 · Cited by 59
“Quantitatively, we use the computation-to-communication (C2C) ratio [32] (which is also named “memory intensity” in Reference [28]) to justify whether a computation kernel is computation/communication bounded.”
MethodSelf-citationBoosting the Performance of CNN Accelerators with Dynamic Fine-Grained Channel Gating
Hua, W., Zhou, Y., De, C., Zhang, Z., & Suh, G. E. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358283 · Cited by 56
“If the network contains convolutional filters of different sizes, techniques similar to [15] can be used to choose an optimal value of P .”
MethodA survey of FPGA design for AI era
Li, Z., Zhang, Y., Wang, J., & Lai, J. (2020) · Journal of Semiconductors · doi:10.1088/1674-4926/41/2/021402 · Cited by 59
Shuhai: A Tool for Benchmarking High Bandwidth Memory on FPGAs
Huang, H., Wang, Z., Zhang, J., He, Z., Wu, C., Xiao, J., & Alonso, G. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3075765 · Cited by 59
“HBM has the potential to be a game-changing feature by allowing FPGAs to provide significantly higher performance for memory-and compute-bound applications like database engines [2] or deep learning inference [3].”
BackgroundRAPIDNN: In-Memory Deep Neural Network Acceleration FrameworkPreprint
Imani, M., Samragh, M., Kim, Y., Gupta, S., Koushanfar, F., & Rosing, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05794 · Cited by 59
Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework ICCAD Special Session Paper
Shafique, M., Marchisio, A., Wicaksana Putra, R. V., & Hanif, M. A. (2021) · 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD) · doi:10.1109/iccad51958.2021.9643539 · Cited by 58
“Toward this, previous works have proposed (1) model compression through pruning [15]–[19] and quantization [19] [20]–[22], and (2) data partitioning and scheduling schemes [33]–[36].”
MethodMAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores
Kao, S.-C., & Krishna, T. (2022) · 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00065 · Cited by 58
ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network
Gschwend, D. (2020) · arXiv · doi:10.48550/arxiv.2005.06892 · Cited by 58
Atomic Dataflow based Graph-Level Workload Orchestration for Scalable DNN Accelerators
Zheng, S., Zhang, X., Liu, L., Wei, S., & Yin, S. (2022) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00042 · Cited by 58
Double MAC: Doubling the performance of convolutional neural networks on modern FPGAs
Nguyen, D. T., Kim, D. W., & Lee, J. (2017) · Design, Automation, and Test in Europe · doi:10.23919/date.2017.7927113 · Cited by 58
“Using DSP blocks is critical, since most CNN implementations on FPGAs rely on DSP blocks for MAC operations [1], and therefore being able to perform two MACs with one DSP block essentially means free 2X improvement in computation throughput.”
BackgroundMethodResultInfluentialShah, N., Chaudhari, P., & Varghese, K. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2815085 · Cited by 58
FPGA-based CNN inference accelerator synthesized from multi-threaded C software
Kim, J. H., Grady, B., Lian, R., Brothers, J., & Anderson, J. H. (2017) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc.2017.8226056 · Cited by 58
“[14] applied Xilinx’s Vivado HLS to synthesize an accelerator for the convolution operation, achieving 61 GFLOPS peak performance.”
MethodEfficient Object Detection Framework and Hardware Architecture for Remote Sensing Images
Li, L., Zhang, S., & Wu, J. (2019) · Remote Sensing · doi:10.3390/rs11202376 · Cited by 58
“designed a CNN accelerator based on the adder tree structure by quantitative analysis of memory bandwidth required for throughput [41].”
MethodCaffePresso: an optimized library for deep learning on embedded accelerator-based platforms
Hegde, G., Siddhartha, Ramasamy, N., & Kapre, N. (2016) · Proceedings of the International Conference on Compilers, Architectures and Synthesis for Embedded Systems · doi:10.1145/2968455.2968511 · Cited by 46
“Unlike solutions (FPGAs [10, 14, 7], GPUs [13]) using abundant hardware resources, embedded solutions are severely constrained by the limited capacity of logic, on-chip memory and communication bandwidth.”
BackgroundAutomatic Compiler Based FPGA Accelerator for CNN Training
Venkataramanaiah, S. K., Ma, Y., Yin, S., Nurvithadhi, E., Dasu, A. R., Cao, Y. K., & Seo, J.-S. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00034 · Cited by 57
“Double buffering scheme is employed to hide the memory access latency [3], which reduced the latency of weight update layers by 11%.”
BackgroundMethodHardware Implementation of Neural Self-Interference Cancellation
Kurzo, Y., Kristensen, A. T., Burg, A. P., & Balatsoukas‐Stimming, A. (2020) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2020.2992370 · Cited by 57
“Existing NN hardware accelerators, such as [26], [27], mainly target applications where both the size of the NN and the number of inputs is very large, and where producing a few tens of outputs per second is sufficient.”
BackgroundAttention-based Convolutional Autoencoders for 3D-Variational Data Assimilation
Mack, J., Arcucci, R., Molina-Solana, M., & Guo, Y. (2020) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2020.113291 · Cited by 57
“There are also lower-cost, lower-power options such as FPGAs [78].”
BackgroundToward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs
Liu, S., Fan, H., Ferianc, M., Niu, X., Shi, H.-F., & Luk, W. W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3055240 · Cited by 57
“From the hardware level perspective, researchers [25]–[27] have proposed: 1) a quantization method, which captures the specialty of FPGAs with capability of custom precision support to save computation resources and 2) loop unrolling strategies for multiple parallel processing.”
MethodEfficient depthwise separable convolution accelerator for classification and UAV object detection
Li, G., Zhang, J., Zhang, M., Wu, R., Cao, X., & Liu, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.071 · Cited by 57
TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
Gao, M., Pu, J., Yang, X., Horowitz, M., & Kozyrakis, C. (2017) · ACM SIGPLAN Notices · doi:10.1145/3093336.3037702 · Cited by 57
Machine learning on FPGAs to face the IoT revolution
Zhang, X., Ramachandran, A., Zhuge, C., He, D., Zuo, W., Cheng, Z., Rupnow, K., & Chen, D. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203875 · Cited by 57
Optimally Scheduling CNN Convolutions for Efficient Memory AccessPreprint
Stoutchinin, A., Conti, F., & Benini, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.01492 · Cited by 56
Rakin, A. S., Luo, Y., Xu, X., & Fan, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03006 · Cited by 59
“Practically, DNN computation is usually accomplished in a layer-by-layer style, i.e., input data like image and DNN model parameters of different layers are usually loaded and processed separately [70–72].”
MethodAn FPGA-based MobileNet Accelerator Considering Network Structure Characteristics
Yan, S., Liu, Z., Wang, Y., Zeng, C., Liu, Q., Cheng, B., & Cheung, R. C. C. (2021) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl53798.2021.00011 · Cited by 56
Algorithm/Hardware Codesign for Real-Time On-Satellite CNN-Based Ship Detection in SAR Imagery
Yang, G., Lei, J., Xie, W., Fang, Z., Li, Y., Wang, J., & Zhang, X. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3161499 · Cited by 56
“For instance, Zhang et al. [42], [43] used the roofline model [49] to obtain the optimal parameter configuration for each layer by balancing computation and communication.”
BackgroundMethodFPGA-based implementation of classification techniques: A survey
Saidi, A., Othman, S. B., Dhouibi, M., & Saoud, S. B. (2021) · Integration · doi:10.1016/j.vlsi.2021.08.004 · Cited by 55
AHA: An Agile Approach to the Design of Coarse-Grained Reconfigurable Accelerators and Compilers
Koul, K., Melchert, J., Sreedhar, K., Truong, L., Nyengele, G., Zhang, K., Liu, Q., Setter, J., Chen, P., Mei, Y., et al. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3534933 · Cited by 55
“DNNWeaver uses Caffe to generate synthesizable designs with several layer types and [59] uses Caffe [21] to optimize DNN blocking using double buffer structures.”
MethodDeng, K., Zhao, D. Y., Han, Q., Wang, S., Zhang, Z., Zhou, A., & Ma, H. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3550298 · Cited by 55
“There is also an extensive line of work focusing on model acceleration, including neural network compression methods [24, 30, 42], hardware-based methods [27, 65, 71, 78, 80] and software-based methods [18, 47, 74].”
BackgroundDomain-Specific Architectures: Research Problems and Promising Approaches
Krishnakumar, A., Ogras, U. Y., Mărculescu, R., Kishinevsky, M., & Mudge, T. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3563946 · Cited by 56
“FPGA-based parallel implementations have accelerated a wide range of operations, from matrix multiplications [52] to Fourier transforms [149], and are currently extensively deployed to enable high-throughput and low-latency neural network inference computations [159, 190].”
Metadata
- Authors
- Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J.
- Affiliations
- Peking University (CN)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
- Publisher
- ACM
- Published
- 2015-02-22
- Pages
- 161-170
- OpenAlex
- W2094756095
- Cited by
- 2,158
- References
- 17
- Citations by year
- 2026: 78 · 2025: 116 · 2024: 145 · 2023: 162 · 2022: 233 · 2021: 253 · 2020: 258 · 2019: 303 · 2018: 281 · 2017: 221 · 2016: 90 · 2015: 15
- FWCI
- 133.92
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) (publishedVersion)
- Funders
- C-FAR
- NSF China (61202072)
- National High Technology Research and Development Program of China (2012AA010902)
- RFDP (20110001110099)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
- Keywords
- FPGA accelerator, deep convolutional neural networks, roofline model, memory bandwidth, loop tiling, loop transformation, FPGA resource utilization, image recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.