Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Convolutional neural network (CNN) has been widely employed for image recognition because it can achieve high accuracy by emulating behavior of optic nerves in living creatures. Recently, rapid growth of modern applications based on deep learning algorithms has further improved research and implementations. Especially, various accelerators for deep CNN have been proposed based on FPGA platform because it has advantages of high performance, reconfigurability, and fast development round, etc. Although current FPGA accelerators have demonstrated better performance over generic processors, the accelerator design space has not been well exploited. One critical problem is that the computation throughput may not well match the memory bandwidth provided an FPGA platform. Consequently, existing approaches cannot achieve best performance due to under-utilization of either logic resource or memory bandwidth. At the same time, the increasing complexity and scalability of deep learning applications aggravate this problem. In order to overcome this problem, we propose an analytical design scheme using the roofline model. For any solution of a CNN design, we quantitatively analyze its computing throughput and required memory bandwidth using various optimization techniques, such as loop tiling and transformation. Then, with the help of rooine model, we can identify the solution with best performance and lowest FPGA resource requirement. As a case study, we implement a CNN accelerator on a VC707 FPGA board and compare it to previous approaches. Our implementation achieves a peak performance of 61.62 GFLOPS under 100MHz working frequency, which outperform previous approaches significantly.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another option is to use field programable gate arrays (FPGAs), which consume much less energy than GPUs while providing an intermediate computing efficiency between CPUs and GPUs13–17.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [15]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“Figure 2 shows a real-life CNN application, taken from [9].”
BackgroundMethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“The algorithm receives three 224x224 input images that are from an original 256x256 three-channel RGB image.”
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992 · cited 1×
“Convolutional neural network (CNN), a well-known deep learning architecture extended from artificial neural network, has been extensively adopted in various applications, which include video surveillance, mobile robot vision, image search engine in data centers, etc [6] [7] [8] [10] [14].”
BackgroundMinimizing Computation in Convolutional Neural Networks
Cong, J., & Xiao, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11179-7_36 · Cited by 372 · cited 1×
“In the feedforward computation perspective, a previous study [5] proved that convolution operations will occupy over 90% of the computation time.”
BackgroundSelf-citationCNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343 · cited 8×
“Among these approaches, FPGA based accelerators have attracted more and more attention of researchers because they have advantages of good performance, high energy efficiency, fast development round, and capability of reconfiguration [1] [2] [3] [6] [12] [14].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- D. Aysegul , J. Jonghoon , G. Vinayak , K. Bharadwaj , C. Alfredo , M. Berin , and C. Eugenio . Accelerating deep neural networks on mobile processor with embedded programmable logic . In NIPS 2013 . IEEE, 2013 . D. Aysegul, J. Jonghoon, G. Vinayak, K. Bharadwaj, C. Alfredo, M. Berin, and C. Eugenio. Accelerating deep neural networks on mobile processor with embedded programmable logic. In NIPS 2013. IEEE, 2013.
Cited by
Items 401–500 of 1,002
Field-Programmable Gate Array Architecture for Deep Learning: Survey and Future Directions
Boutros, A., Arora, A., & Betz, V. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3623023 · Cited by 56
“In this case, the FPGA accelerator is usually hand-crafted and optimized for a specific DL model or a group of similar models [40], [41], [42], [43].”
Tensor Casting: Co-Designing Algorithm-Architecture for Personalized Recommendation Training
Kwon, Y., Lee, Y., & Rhu, M. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00029 · Cited by 56
“d computation using custom-designed architectures for both training and inference [3], [9], [11], [12], [13], [26], [30], [33], [42], [44], [45], [48], [59], [60], [63], [64], [68], [72], [73], [74], [75], [76]. Unlike these prior art, DNN-based recommendation models are a new type of ML workload that has only recentl…”
MethodGNNear: Accelerating Full-Batch Training of Graph Neural Networks with near-Memory Processing
Zhou, Z., Li, C., Wei, X., Wang, X., & Sun, G. (2022) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1145/3559009.3569670 · Cited by 62
“As shown in Table 3, Reduce operations show a much lower arithmetic intensity and worse data locality than Update and Others, which is inefficient to accelerate with traditional neural network accelerators [10, 12, 13, 78, 115] optimized for computation-intensive workloads.”
BackgroundSelf-citationFPGA-based convolutional neural network accelerator design using high level synthesize
Ghaffari, S., & Sharifian, S. (2016) · International Conference on Signal Processing and Intelligent Systems (ICSPIS) · doi:10.1109/icspis.2016.7869873 · Cited by 54
“Yongmei et al work [16] has 156 (25x6+6) parameters in first CNN and 312 (25x12+12) parameters in the second CNN layer and is much less than our network.”
BackgroundMethodCustomizing Neural Networks for Efficient FPGA Implementation
Samragh, M., Ghasemzadeh, M., & Koushanfar, F. (2017) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.43 · Cited by 54
“Since FPGAs are promising platforms for high-throughput and power efficient implementation of parallel algorithms, efficient FPGA realization of DNNs on single FPGA chips has attracted researchers’ attention [10], [11], [12].”
BackgroundLow-Complexity Approximate Convolutional Neural Networks
Cintra, R. J., Duffner, S., García, C., & Leite, A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2815435 · Cited by 54
“Finally, Zhang et al. [89] propose a FPGA design strategy and algorithmic enhancements to optimize the computational throughput and memory bandwidth for any given ConvNet architecture.”
BackgroundLaius: An 8-Bit Fixed-Point CNN Hardware Inference Engine
Li, Z., Wang, L., Guo, S., Deng, Y. L., Dou, Q., Zhou, H., & Lu, W. (2017) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa/iucc.2017.00030 · Cited by 54
“In the work of [1] [2], the author uses the Roofline model as a guide to form a better trade-off among bandwidth, throughput and resource utilization.”
MethodGNA: Reconfigurable and Efficient Architecture for Generative Network Acceleration
Yan, J., Yin, S., Tu, F., Liu, L., & Wei, S. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2857258 · Cited by 54
Mousouliotis, P., & Petrou, L. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.102990 · Cited by 54
End-to-End Optimization of Deep Learning Applications
Sohrabizadeh, A., Wang, J., & Cong, J. (2020) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3373087.3375321 · Cited by 54
“[26] showed that the CTC ratio of a single convolution layer varies with different tiling factors.”
BackgroundSelf-citationAn Overview of FPGA Based Deep Learning Accelerators: Challenges and Opportunities
Wang, T., Wang, C., Zhou, X., & Chen, H. (2019) · 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc/smartcity/dss.2019.00229 · Cited by 53
“By the paper[6], the double-buffer mechanism is used to optimize the memory access, and the CTC is further improved.”
BackgroundMethodInfluentialHardware Acceleration of a Generalized Fast 2-D Convolution Method for Deep Neural Networks
Ansari, A., & Ogunfunmi, T. (2022) · IEEE Access · doi:10.1109/access.2022.3149505 · Cited by 54
“In [22] we focused on increasing the computational throughput using a novel processing unit design called nested processing element (NPE) using the variable unroll parameters calculated in [17] to achieve a throughput enhancement of 94%.”
BackgroundMethodDeep Learning Inferencing with High-performance Hardware Accelerators
Kljucaric, L., & George, A. D. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3594221 · Cited by 53
Review of neural network model acceleration techniques based on FPGA platforms
Liu, F., Li, H., Hu, W., & He, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128511 · Cited by 54
HLScope+,: Fast and accurate performance estimation for FPGA HLS
Choi, Y., Zhang, P., Li, P., & Cong, J. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203844 · Cited by 53
“The benchmark we used includes quicksort [10], Cholesky [20], convolutional neural network [28], matrix multiplication [13], logistic regression [3], decompression [17], and compression [9].”
MethodSelf-citationAlgorithm-hardware Co-design of Attention Mechanism on FPGA Devices
Zhang, X., Wu, Y., Zhou, P., Tang, X., & Hu, J. (2021) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3477002 · Cited by 52
“Such mechanism is widely adopted in Convolution Neural Network (CNN) accelerator designs [13–20].”
BackgroundGerlinghoff, D., Wang, Z., Gu, X., Goh, R. S. M., & Luo, T. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3128945 · Cited by 52
“Optimization can be applied to reduce data movement and necessary memory capacity, as done before for ANNs by [46], [47].”
BackgroundFadhel, M. A., Alzubaidi, L., Gu, Y., Santamaría, J., & Duan, Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18304-x · Cited by 52
“Zhang et al. are the closest competitor to realize another significant achievement: processing 46 images/second on Virtex 7485 T without reducing the power consumption [12].”
HIDA: A Hierarchical Dataflow Compiler for High-Level Synthesis
Ye, H., Jun, H., & Chen, D. (2024) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3617232.3624850 · Cited by 52
“The listed parallel factors are selected with heuristics [76] and implemented with manual loop tiling and loop unroll directive insertion.”
MethodSynergy: An HW/SW Framework for High Throughput CNNs on Embedded Heterogeneous SoC
Zhong, G., Dubey, A., Tan, C., & Mitra, T. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3301278 · Cited by 52
“[3][6][13][17][20][21][23][24] mainly focus on exploiting intra-frame parallelism within layers and execute layers in a CNN in sequence, ignoring inter-frame parallelism across layers.”
BackgroundMethodInfluentialFixed-Point Convolutional Neural Network for Real-Time Video Processing in FPGA
Solovyev, R. A., Kustov, A. G., Telpukhov, D. V., Сергеевич, Р. В., & Kalinin, A. A. (2019) · IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering · doi:10.1109/eiconrus.2019.8656778 · Cited by 52
“[6] quantitatively analyzed computing throughput and required memory bandwidth for various CNNs using optimization techniques, such as loop tiling and transformation.”
MethodIntermittent-Aware Neural Architecture Search
Mendis, H. R., Kang, C.-K., & Hsiu, P.-C. (2021) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3476995 · Cited by 52
“, [68]) is used to analytically estimate the inference latency of a candidate design without requiring real deployment on the target platform, considering the computation and data access costs related to inference execution.”
BackgroundMethodInfluentialNajmaei, S., Glasmann, A., Schroeder, M. A., Sarney, W. L., Chin, M. L., & Potrepka, D. M. (2022) · Materials Today · doi:10.1016/j.mattod.2022.08.017 · Cited by 51
Design space exploration of FPGA accelerators for convolutional neural networks
Rahman, A., Oh, S., Lee, J., & Choi, K. (2017) · Design, Automation, and Test in Europe · doi:10.23919/date.2017.7927162 · Cited by 51
“The authors of [3] present a CNN architecture for FPGAs and a parameter optimization methodology based on the well-known roofline model, which is one method to help balance computation time and data transfer time.”
BackgroundMethodResultInfluentialNeuromorphic computing's yesterday, today, and tomorrow – an evolutional view
Chen, Y., Li, H. H., Wu, C., Song, C., Li, S. K., Min, C., Cheng, H.-P., Wen, W., & Liu, X. (2017) · Integration · doi:10.1016/j.vlsi.2017.11.001 · Cited by 51
Kurzo, Y., Burg, A. P., & Balatsoukas‐Stimming, A. (2018) · Asilomar Conference on Signals, Systems and Computers (ACSSC) · doi:10.1109/acssc.2018.8645295 · Cited by 51
“Existing NN hardware accelerators, such as [10], [11], mainly target applications where both the size of the NN and the number of inputs is very large, and where producing a a few tens of outputs per second is sufficient.”
BackgroundShortcut Mining: Exploiting Cross-Layer Shortcut Reuse in DCNN Accelerators
Mazreah, A. A., & Chen, L. (2019) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2019.00030 · Cited by 51
“For each pair, the tool estimates the off-chip traffic, bank access delay, and memory utilization based on [29][42][26] and the physical characteristics of the tracking arrays.”
BackgroundMethodInfluentialFrequency Improvement of Systolic Array-Based CNNs on FPGAs
Zhang, J., Zhang, W., Luo, G., Wei, X., Liang, Y. E., & Cong, J. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702071 · Cited by 51
“And FPGA is a commodity off-the-shelf technology to implement customized computing and applies to an increased amount of CNN designs [1], [2], [3], [4], [5], [6].”
MethodSelf-citationSqueezeFlow: A Sparse CNN Accelerator Exploiting Concise Convolution Rules
Li, J., Jiang, S., Gong, S., Wu, J., Yan, J., Yan, G., & Li, X. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2924215 · Cited by 51
“[18] focuses on the balance between computing resource and memory bandwidth for CVLs.”
BackgroundZokaee, F., Lou, Q., Youngblood, N., Liu, W., Xie, Y., & Jiang, L. (2020) · Design, Automation, and Test in Europe · doi:10.23919/date48585.2020.9116494 · Cited by 51
“Instead of power-hungry GPUs, ASIC [7], FPGA [31], and ReRAM [25]-based accelerators are built to more energy-efficiently process CNN inferences.”
BackgroundMethodA Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGAPreprint
Zhang, X., Das, S., Neopane, O., & Kreutz-Delgado, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02583 · Cited by 51
Security of Cloud FPGAs: A SurveyPreprint
Jin, C., Gohil, V., Karri, R., & Rajendran, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04867 · Cited by 51
“FPGA accelerators can boost the performance of deep learning technology, and thus accelerate numerous applications and services ranging from database management to artificial intelligence [59, 140].”
BackgroundAccelerator Implementation of Lenet-5 Convolution Neural Network Based on FPGA with HLS
Dai, R., & Tang, Y. (2019) · International Conference on Circuits, Systems and Simulation (ICCSS) · doi:10.1109/cirsyssim.2019.8935599 · Cited by 50
“As shown in the following figure, first configure the data in DDR through ARM, then configure DMA to read the data stream through AXI stream, and DMA input the data stream into CNN IP, ARM to read the recognition result of convolution neural network through AXI lite bus [4].”
MethodSC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing
Ren, A., Li, Z., Ding, C., Qiu, Q., Wang, Y., Li, J., Qian, X., & Yuan, B. (2017) · ACM SIGOPS Operating Systems Review · doi:10.1145/3093315.3037746 · Cited by 50
Memory-Efficient Deformable Convolution Based Joint Denoising and Demosaicing for UHD Images
Guan, J., Lai, R., Yang, L., Li, Y., Li, H., Feng, L., Yang, Y., & Gu, L. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3182990 · Cited by 50
“In particular, field-programmable gate array (FPGA) based CNN accelerators are more energy efficient than GPUs and can perform more massive parallel processing than CPUs [14].”
BackgroundNegrov, D., Karandashev, I., Shakirov, V., Matveyev, Y., Dunin-Barkowski, W., & Zenkevich, A. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.10.061 · Cited by 49
“Among the approaches existing today, the one coming close to take full advantage of huge parallelism of ANNs is an FPGA-based implementation, such as described in [3].”
Liao, J.-X., Wei, S.-L., Xie, C.-L., Zeng, T., Sun, J., Zhang, S., Zhang, X., & Fan, F. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3346517 · Cited by 49
“As for the latter, some prior researchers have designed FPGA-based accelerators for CNNs and successfully deployed CNNs on Virtex series FPGA [24], [25], [26].”
MethodUltra-compact multi-task processor based on in-memory optical computing
Liu, W., Huang, Y., Sun, R. C., Fu, T., Yang, S., & Chen, H. (2025) · Light Science & Applications · doi:10.1038/s41377-025-01814-0 · Cited by 49
Communication Lower Bound in Convolution Accelerators
Chen, X., Han, Y., & Wang, Y. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00050 · Cited by 49
“There are various approaches to optimize the dataflow: 1) designing an elaborate dataflow [5], [7]– [18], 2) selecting the best dataflow from several candidates [19]–[23], and 3) design space exploration (DSE) [24]–[32].”
MethodHigh-Throughput Convolutional Neural Network on an FPGA by Customized JPEG Compression
Nakahara, H., Que, Z., & Luk, W. W. (2020) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm48280.2020.00010 · Cited by 49
“However, these FC layers require many parameters, and this causes an over-fitting problem [47].”
BackgroundWu, N., Jiang, T., Zhang, L., Zhou, F., & Ge, F. (2020) · Electronics · doi:10.3390/electronics9061005 · Cited by 49
“In Reference [5,6], most of the constituent units in the CNN network are implemented, which need to consume more hardware resources while obtaining higher computing acceleration performance, and cannot meet the resource-limited needs of nodes in the IoT systems.”
BackgroundOvercoming resource underutilization in spatial CNN accelerators
Shen, Y., Ferdman, M., & Milder, P. A. (2016) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2016.7577315 · Cited by 48
“We report results using single-precision floating point and a 100 MHz clock in order to compare directly with [4], although Vivado HLS can reach much higher clock frequencies for our design.”
BackgroundMethodInfluentialAn efficient implementation of 2D convolution in CNN
Chang, J., & Sha, J. (2016) · IEICE Electronics Express · doi:10.1587/elex.13.20161134 · Cited by 48
“[9] improves the efficiency of FPGA-based CNN design by quantitatively analyzing the relationship between computing throughout and memory bandwidth.”
Deep reuse: streamline CNN inference on the fly via coarse-grained computation reuse
Ning, L., & Shen, X. (2019) · International Conference on Supercomputing · doi:10.1145/3330345.3330384 · Cited by 48
“FPGA based accelerators [1, 19, 22, 24, 25, 36] also show great potential on accelerating DNN inference.”
BackgroundBenchIP: Benchmarking Intelligence Processors
Tao, J.-H., Du, Z., Guo, Q., Lan, H., Zhang, L., Zhou, S., Xu, L., Liu, C., Liu, H.-F., Tang, S., et al. (2018) · Journal of Computer Science and Technology · doi:10.1007/s11390-018-1805-8 · Cited by 49
An FPGA-based processor for training convolutional neural networks
Liu, Z., Dou, Y., Jiang, J., Wang, Q., & Chow, P. (2017) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2017.8280142 · Cited by 48
“Efficient computation engines are designed and various parallel strategies are explored [4] [5].”
BackgroundHardware-Software Co-Design for an Analog-Digital Accelerator for Machine Learning
Ambrosi, J., Ankit, A., Antunes, R., Chalamalasetti, S. R., Chatterjee, S., Hajj, I. E., Fachini, G. J. D. A., Faraboschi, P., Foltín, M., Huang, S., et al. (2018) · IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc.2018.8638612 · Cited by 48
“There have been many digital-only accelerators designed for machine learning applications [13], [14], [15], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67].”
BackgroundShen, J., Huang, Y. R., Wen, M., & Zhang, C. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2019.2912894 · Cited by 47
“A representative work by Zhang et al. [4] proposed a roofline model to maximize computational resources on FPGA under the memory bandwidth limitations.”
BackgroundMethodResultInfluentialSelf-citationDepthNet: Real-Time LiDAR Point Cloud Depth Completion for Autonomous Vehicles
Bai, L., Zhao, Y., Elhousni, M., & Huang, X. (2020) · IEEE Access · doi:10.1109/access.2020.3045681 · Cited by 47
“In [9], the authors proposed a novel architecture for process element array.”
BackgroundA Mixed-Pruning Based Framework for Embedded Convolutional Neural Network Acceleration
Chang, X., Pan, H., Lin, W., & Gao, H. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2020.3048260 · Cited by 47
Democratizing Domain-Specific Computing
Chi, Y., Qiao, W., Sohrabizadeh, A., Wang, J., & Cong, J. (2022) · Communications of the ACM · doi:10.1145/3524108 · Cited by 46
“One of the earliest, also probably the most cited FPGA-based deep learning accelerator is the one published in early 2015 [19].”
BackgroundSelf-citationA survey of neural network accelerators
Li, Z., Wang, Y., Zhi, T., & Chen, T. (2017) · Frontiers of Computer Science · doi:10.1007/s11704-016-6159-1 · Cited by 46
Performance Evaluation and Optimization of HBM-Enabled GPU for Data-Intensive Applications
Zhu, M., Zhuo, Y., Wang, C., Chen, W., & Xie, Y. (2018) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2791442 · Cited by 46
“Microsoft has proposed a heterogeneous architecture with GPU/FPGA installed for the both applications in their new datacenter design [13].”
InfluentialTowards Efficient Convolutional Neural Network for Domain-Specific Applications on FPGA
Zhao, R., Ng, H.-C., Luk, W. W., & Niu, X. (2018) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2018.00033 · Cited by 46
Hardware Trojan in FPGA CNN Accelerator
Ye, J., Hu, Y. H., & Li, X. (2018) · Proceedings - Asian Test Symposium/Proceedings · doi:10.1109/ats.2018.00024 · Cited by 46
Face Recognition with Hybrid Efficient Convolution Algorithms on FPGAs
Zhuge, C., Liu, X., Zhang, X., Gummadi, S., Xiong, J., & Chen, D. (2018) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3194554.3194597 · Cited by 46
“Prior FPGA works [9, 10] mostly focus on optimizing general convolution (CONV) layers.”
Putra, R. V. W., Hanif, M. A., & Shafique, M. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2021.3060509 · Cited by 48
“We consider the layer partitioning approach in the optimization problem, as it is employed by many CNN accelerators, such as [3], [7], and [12].”
MethodInfluentialOptimizing FPGA-based convolutional neural networks accelerator for image super-resolution
Chang, J.-W., & Kang, S. (2018) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2018.8297347 · Cited by 46
“[5] propose a tiling technique that effectively parallelizes the dimensions of the matrix.”
BackgroundMethodInfluentialKim, N. S., Chen, D., Xiong, J., & Hwu, W. (2017) · IEEE Micro · doi:10.1109/mm.2017.3211105 · Cited by 46
“Since such machine-learning algorithms are memory-intensive, the performance of running them on traditional general-purpose processors with their memory systems is often limited by the available main-memory bandwidth [10].”
BackgroundAn FPGA-based online reconfigurable CNN edge computing device for object detection
Wang, Y., Liao, Y., Yang, J., Wang, H., Zhao, Y., Zhang, C., Xiao, B., Xu, F., Gao, Y., Xu, M., et al. (2023) · Microelectronics Journal · doi:10.1016/j.mejo.2023.105805 · Cited by 45
Self-citationLow Complexity Multiply-Accumulate Units for Convolutional Neural Networks with Weight-Sharing
Garland, J., & Gregg, D. (2018) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3233300 · Cited by 45
“The loops can be unrolled into parallel MAC units and implemented in hardware [Zhang et al. 2015] to accelerate the convolution.”
MethodHeterogeneous FPGA-Based Cost-Optimal Design for Timing-Constrained CNNs
Jiang, W., Sha, E. H. ‐., Zhuge, Q., Yang, L., Chen, X., & Hu, J. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2857098 · Cited by 45
“We compare the proposed BLAST and BLAST-R with three state-of-the-art techniques: 1) a uniform PE on one FPGA [20];”
BackgroundMethodResultInfluentialWhen Neural Architecture Search Meets Hardware Implementation: from Hardware Awareness to Co-Design
Zhang, X., Jiang, W., Shi, Y., & Hu, J. (2019) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2019.00014 · Cited by 45
“[1]”
BackgroundMethodInfluentialSophisticated deep learning with on-chip optical diffractive tensor processing
Huang, Y., Fu, T., Huang, H., Yang, S., & Chen, H. (2023) · Photonics Research · doi:10.1364/prj.484662 · Cited by 44
Li, J., Shen, G., Zhao, D., Zhang, Q., & Zeng, Y. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3380550 · Cited by 45
“The permutation of loop order does not alter the computation results, yet it does influence data locality and data reuse opportunities [22].”
BackgroundHuman action recognition using support vector machines and 3D convolutional neural networks
Latah, M. (2017) · International Journal of Advances in Intelligent Informatics · doi:10.26555/ijain.v3i1.89 · Cited by 44
“Each feature map will be convolved using a shifting window with a K x K kernel in order to produce the one pixel in one output feature map [33].”
BackgroundMethodKästner, F., Benedikt, J., Kautz, F., Hübner, M., & Corradi, G. (2018) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2018.00031 · Cited by 44
“[10] to apply this concept performing a design space exploration of beneficial CNN hardware implementations.”
MethodDeepFense: online accelerated defense against adversarial deep learning
Rouhani, B. D., Samragh, M., Javaheripi, M., Javidi, T., & Koushanfar, F. (2018) · doi:10.1145/3240765.3240791 · Cited by 44
“[12–16], none of them have been particularly optimized for in-time defense against adversarial inputs.”
BackgroundServing deep neural networks at the cloud edge for vision applications on mobile platforms
Fang, Z., Hong, D., & Gupta, R. K. (2019) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3304109.3306221 · Cited by 45
“, designing lightweight models [28, 30, 45], pruning and quantizing models [17, 19, 46], and using hardware accelerators [8, 38, 50].”
MethodOctCNN: A High Throughput FPGA Accelerator for CNNs using Octave Convolution Algorithm
Lou, W., Gong, L., Wang, C., Du, Z., & Zhou, X. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3110413 · Cited by 44
AdderNet and its Minimalist Hardware Design for Energy-Efficient Artificial IntelligencePreprint
Wang, Y., Huang, M., Han, K., Chen, H., Zhang, W., Xu, C., & Tao, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10015 · Cited by 44
“Finally, we thoroughly analyze the practical power and chip area consumption of AdderNet and CNN on FPGA.”
BackgroundMethodInfluentialChallenges and Opportunities to Enable Large-Scale Computing via Heterogeneous Chiplets
Yang, Z., Ji, S., Chen, X., Zhuang, J., Zhang, W., Jani, D., & Zhou, P. (2024) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac58780.2024.10473961 · Cited by 43
“We use arithmetic intensity [13] to characterize the algorithms’ data reuse ratio and computation-to-communication (CTC) ratio [14] to characterize the execution time between computation and communication.”
MethodSPARCNet: A Hardware Accelerator for Efficient Deployment of Sparse Convolutional Networks
Page, A., Jafari, A., Shea, C., & Mohsenin, T. (2017) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3005448 · Cited by 43
Logic Synthesis for Established and Emerging Computing
Testa, E., Soeken, M., Amar, L. G., & Micheli, G. D. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2869760 · Cited by 43
“Moreover, new specialized circuits can be used as engines for computational solvers, thus enabling a virtuous cycle to achieve increasingly higher quality hardware [4].”
BackgroundCollaborative learning between cloud and end devices: an empirical study on location prediction
Lu, Y., Shu, Y., Tan, X., Liu, Y., Zhou, M., Chen, Q., & Pei, D. (2019) · ACM/IEEE Symposium on Edge Computing (SEC) · doi:10.1145/3318216.3363304 · Cited by 44
Putra, R. V. W., Hanif, M. A., & Shafique, M. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218672 · Cited by 44
“The State-of-the-Art and Limitations Previous works have proposed different techniques to reduce the DRAM access energy, by minimizing the number of DRAM accesses [3] [14] [15].”
MethodA Binarized Neural Network Approach to Accelerate in-Vehicle Network Intrusion Detection
Zhang, L., Yan, X., & Ma, D. (2022) · IEEE Access · doi:10.1109/access.2022.3208091 · Cited by 44
“To bridge this gap, one of the approaches [69], [70] is to optimize deep learning computation and memory access in FPGAs by efficiently increasing the utilization of the design space.”
Deep Learning on Edge TPUsPreprint
Sun, Y., & Kist, A. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13732 · Cited by 43
Developing FPGA-Based Accelerators for Deep Learning in Reconfigurable Computing Systems
Kumar, D. T. M. S. (2024) · SCCTS Transactions on Reconfigurable Computing · doi:10.31838/rcc/01.01.01 · Cited by 42
Low Complexity Multiply Accumulate Unit for Weight-Sharing Convolutional Neural Networks
Garland, J., & Gregg, D. (2017) · IEEE Computer Architecture Letters · doi:10.1109/lca.2017.2656880 · Cited by 42
“Other research groups have proposed numerous hardware accelerators for CNNs in both FPGA [5], [9] and ASIC [1].”
Li, Y., Liu, Z., Xu, K., Yu, H., & Ren, F. (2017) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3020078.3021786 · Cited by 42
“This eliminates any DRAM access latency and dramatically reduces the energy consumption of the system comparing to the existing work relying on off-chip storage [1] [3] [12].”
BackgroundMotamedi, M., Fong, D. D., & Ghiasi, S. (2017) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3126555 · Cited by 42
“memory bandwidth by using the on-chip memory efficiently [22].”
MethodF-C3D: FPGA-based 3-dimensional convolutional neural network
Fan, H., Niu, X., Liu, Q., & Luk, W. W. (2017) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.23919/fpl.2017.8056779 · Cited by 42
“In recent years, various FPGA-based implementations of 2D CNNs [1][2] and related optimization strategies [3] have been proposed.”
BackgroundInstruction driven cross-layer CNN accelerator with winograd transformation on FPGA
Yu, J., Hu, Y., Ning, X., Qiu, J., Guo, K., Wang, Y., & Yang, H. (2017) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2017.8280147 · Cited by 42
Hardware Implementation and Optimization of Tiny-YOLO Network
Ma, J., Chen, L., & Gao, Z. (2018) · Communications in computer and information science · doi:10.1007/978-981-10-8108-8_21 · Cited by 42
Opportunities for Machine Learning in Electronic Design Automation
Beerel, P. A., & Pedram, M. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351731 · Cited by 42
“Several researchers have investigated architectures to accelerate inference of these networks, that include efficiently networked 2-D arrays of small neural network processors [16], arrays of FPGAs [17], and algorithmic optimizations including weight pruning, quantification, coding [18]–[21].”
BackgroundBaptista, D., Mostafa, S. S., Pereira, L., Sousa, L. A., & Morgado‐Dias, F. (2018) · Energies · doi:10.3390/en11092460 · Cited by 43
“[39] shows an optimized CNN accelerator designed with HLS, reordering and tiling loops, inserting pragmas and organizing external memory transfers.”
MethodLerGAN: A Zero-Free, Low Data Movement and PIM-Based GAN Architecture
Mao, H., Song, M., Li, T., Dai, Y., & Shu, J. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00060 · Cited by 42
“NN accelerators Many recent works accelerate NN based on FPGAs [67] [45] [69] [57] [3] and ASICs [18] [23] [68] [40] [12] [43] [55] [44] [2].”
BackgroundDeep Neural Network Hardware Implementation Based on Stacked Sparse Autoencoder
Coutinho, M. G. F., Torquato, M. F., & Fernandes, M. A. C. (2019) · IEEE Access · doi:10.1109/access.2019.2907261 · Cited by 42
“As presented in Table 17, the speedup is higher with respect to cases based onHLS, [36] and [12].”
BackgroundMethodResultInfluentialAdaptation of Convolution and Batch Normalization Layer for CNN Implementation on FPGA
Sledević, T. (2019) · IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream) · doi:10.1109/estream.2019.8732160 · Cited by 42
“The results shows, that FPGA gives 177 times better performance in comparison with ARM processor [12].”
MethodInfluentialReal-Time SSDLite Object Detection on FPGA
Kim, S., Na, S., Kong, B. Y., Choi, J., & Park, I. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2021.3064639 · Cited by 42
TelaMalloc: Efficient On-Chip Memory Allocation for Production Machine Learning Accelerators
Maas, M., Beaugnon, U., Chauhan, A., & Ilbeyi, B. (2022) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3567955.3567961 · Cited by 42
“DSP [12] is a canonical example of this type of scenario, but it may also apply to SRAM sizing in HLS [53], BRAM management in time-multiplexed FPGA overlays [40] or CGRAs [54], memory management for scientific computing workloads [57], and memory-trace oblivious computation [41] where timing-predictable scratchpads a…”
MethodChen, C., da Silva, B., Yang, C., Ma, C., Li, J., & Liu, C. (2023) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2023.3299084 · Cited by 41
“This reconfigurable technology has gained interest in ML acceleration because it provides high-computation for parallel algorithms [27], [28], [29] while achieving high-power efficiency [30], [31].”
BackgroundOptimizing CNN Computation Using RISC-V Custom Instruction Sets for Edge Platforms
Wang, S., Wang, X., Xu, Z., Chen, B., Feng, C., Wang, Q., & Ye, T. T. (2024) · IEEE Transactions on Computers · doi:10.1109/tc.2024.3362060 · Cited by 42
“In recent years, many AI applications, such as image/audio/video rendering and processing, have been implemented using CNN architectures on edge computing platforms [9], [10], [11].”
BackgroundVasile, C.-E., Ulmamei, A., & Bîră, C. (2024) · Journal of Imaging · doi:10.3390/jimaging10120298 · Cited by 42
“FPGAs can be designed to process different parts of the matrix multiplication concurrently (pipeline architecture), which optimizes the throughput of fully connected layers [21].”
System-Level Optimization of Accelerator Local Memory for Heterogeneous Systems-on-Chip
Pilato, C., Mantovani, P., Guglielmo, G. D., & Carloni, L. P. (2016) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2016.2611506 · Cited by 41
“For example, hardware accelerators for machine learning applications are increasingly used to identify underlying relations in massive unstructured data [4]–[6].”
BackgroundSong, Z., Liu, Z., & Wang, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11334 · Cited by 41
Real-Time Tone Mapping: A Survey and Cross-Implementation Hardware Benchmark
Ou, Y., Ambalathankandy, P., Shinya, T., Motomura, M., Asai, T., & Ikebe, M. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3060143 · Cited by 41
FPGA-based low-batch training accelerator for modern CNNs featuring high bandwidth memory
Venkataramanaiah, S. K., Suh, H.-S., Yin, S., Nurvitadhi, E., Dasu, A. R., Cao, Y. K., & Seo, J.-S. (2020) · doi:10.1145/3400302.3415643 · Cited by 41
“Many prior works presented low-batch CNN inference on FPGAs and showed large improvements in storage and latency [12–18].”
BackgroundWhen Machine Learning Meets Quantum Computers: A Case Study
Jiang, W., Xiong, J., & Shi, Y. (2021) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3394885.3431629 · Cited by 41
“On the FPGA, work [40] first proposed the tiling based design to accelerate the neural computation, and works [15, 16, 24, 45] gave different designs and extended the implementation to multiple FPGAs.”
Method
Metadata
- Authors
- Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J.
- Affiliations
- Peking University (CN)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
- Publisher
- ACM
- Published
- 2015-02-22
- Pages
- 161-170
- OpenAlex
- W2094756095
- Cited by
- 2,158
- References
- 17
- Citations by year
- 2026: 78 · 2025: 116 · 2024: 145 · 2023: 162 · 2022: 233 · 2021: 253 · 2020: 258 · 2019: 303 · 2018: 281 · 2017: 221 · 2016: 90 · 2015: 15
- FWCI
- 133.92
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) (publishedVersion)
- Funders
- C-FAR
- NSF China (61202072)
- National High Technology Research and Development Program of China (2012AA010902)
- RFDP (20110001110099)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
- Keywords
- FPGA accelerator, deep convolutional neural networks, roofline model, memory bandwidth, loop tiling, loop transformation, FPGA resource utilization, image recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.