Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Convolutional neural network (CNN) has been widely employed for image recognition because it can achieve high accuracy by emulating behavior of optic nerves in living creatures. Recently, rapid growth of modern applications based on deep learning algorithms has further improved research and implementations. Especially, various accelerators for deep CNN have been proposed based on FPGA platform because it has advantages of high performance, reconfigurability, and fast development round, etc. Although current FPGA accelerators have demonstrated better performance over generic processors, the accelerator design space has not been well exploited. One critical problem is that the computation throughput may not well match the memory bandwidth provided an FPGA platform. Consequently, existing approaches cannot achieve best performance due to under-utilization of either logic resource or memory bandwidth. At the same time, the increasing complexity and scalability of deep learning applications aggravate this problem. In order to overcome this problem, we propose an analytical design scheme using the roofline model. For any solution of a CNN design, we quantitatively analyze its computing throughput and required memory bandwidth using various optimization techniques, such as loop tiling and transformation. Then, with the help of rooine model, we can identify the solution with best performance and lowest FPGA resource requirement. As a case study, we implement a CNN accelerator on a VC707 FPGA board and compare it to previous approaches. Our implementation achieves a peak performance of 61.62 GFLOPS under 100MHz working frequency, which outperform previous approaches significantly.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another option is to use field programable gate arrays (FPGAs), which consume much less energy than GPUs while providing an intermediate computing efficiency between CPUs and GPUs13–17.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [15]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“Figure 2 shows a real-life CNN application, taken from [9].”
BackgroundMethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“The algorithm receives three 224x224 input images that are from an original 256x256 three-channel RGB image.”
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Williams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992 · cited 1×
“Convolutional neural network (CNN), a well-known deep learning architecture extended from artificial neural network, has been extensively adopted in various applications, which include video surveillance, mobile robot vision, image search engine in data centers, etc [6] [7] [8] [10] [14].”
BackgroundMinimizing Computation in Convolutional Neural Networks
Cong, J., & Xiao, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11179-7_36 · Cited by 372 · cited 1×
“In the feedforward computation perspective, a previous study [5] proved that convolution operations will occupy over 90% of the computation time.”
BackgroundSelf-citationCNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343 · cited 8×
“Among these approaches, FPGA based accelerators have attracted more and more attention of researchers because they have advantages of good performance, high energy efficiency, fast development round, and capability of reconfiguration [1] [2] [3] [6] [12] [14].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- D. Aysegul , J. Jonghoon , G. Vinayak , K. Bharadwaj , C. Alfredo , M. Berin , and C. Eugenio . Accelerating deep neural networks on mobile processor with embedded programmable logic . In NIPS 2013 . IEEE, 2013 . D. Aysegul, J. Jonghoon, G. Vinayak, K. Bharadwaj, C. Alfredo, M. Berin, and C. Eugenio. Accelerating deep neural networks on mobile processor with embedded programmable logic. In NIPS 2013. IEEE, 2013.
Cited by
Items 201–300 of 1,002
A Survey on Silicon Photonics for Deep Learning
Sunny, F., Taheri, E., Nikdast, M., & Pasricha, S. (2021) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3459009 · Cited by 114
Switched by input: power efficient structure for RRAM-based convolutional neural network
Xia, L., Tang, T., Huangfu, W., Cheng, M., Yin, X., Li, B., Wang, Y., & Yang, H. (2016) · Proceedings of the 53rd Annual Design Automation Conference · doi:10.1145/2897937.2898101 · Cited by 100
Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer
Shen, Y., Ferdman, M., & Milder, P. (2017) · 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.47 · Cited by 106
“[15], [17], [22] use loop tiling to reduce the bandwidth requirements of convolutional and/or fully-connected layers, but without batching.”
BackgroundMethodInfluentialAccelerator-Aware Pruning for Convolutional Neural Networks
Kang, H.-J. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2911674 · Cited by 89
“The architectures proposed in [14]– [17] adopt MWMA-structured PEs, and some of them fetch the activations and weights along the spatial axis [16], [17].”
MethodBlanco‐Filgueira, B., García-Lesta, D., Fernández-Sanjurjo, M., Brea, V. M., & Lopez, P. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2902141 · Cited by 112
FPGA-based Deep Learning Inference Accelerators: Where Are We Standing?
Nechi, A., Groth, L., Mulhem, S., Merchant, F., Buchty, R., & Bereković, M. (2023) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3613963 · Cited by 115
“Zhang et al.[178] use the rooline model to present an analytical approach for optimizing CNN performance on FPGAs.”
MethodResultCamus, V., Mei, L., Enz, C., & Verhelst, M. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2950386 · Cited by 111
“SA-ST arrays contain a 1D vector of PEs per accumulation island whose results are added together, either in a systolic way [24], [25] or through a single-cycle accumulator [26], [27].”
BackgroundLarge‐scale photonic inverse design: computational challenges and breakthroughs
Kang, C., Park, C., Lee, M., Kang, J. S., Jang, M. S., & Chung, H. (2024) · Nanophotonics · doi:10.1515/nanoph-2024-0127 · Cited by 115
“Similarly, the acceleration of solving Maxwell’s equations via computer architecture employs specialized hardware, such as a GPU, digital signal processor (DSP) [124], and field-programmable gate array (FPGA) [125].”
MethodGANAX: A Unified MIMD-SIMD Acceleration for Generative Adversarial Networks
Yazdanbakhsh, A., Falahati, H., Wolfe, P. J., Esmaeilzadeh, H., & Samadi, K. (2018) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2018.00060 · Cited by 109
“A handful of prior work explored the design space of neural network acceleration, which can be categorized into ASICs [15], [16], [18]–[22], [26], [27], [30], [34], [37], [38], [41], [42], FPGA implementations [17], [28], [35], [36], [43], using unconventional devices for acceleration [29], [33], [40], and dataflow op…”
BackgroundGRIP: A Graph Neural Network Accelerator Architecture
Kiningham, K., Levis, P. A., & Ré, C. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3197083 · Cited by 112
“LATED WORK DNN Accelerators. A significant number of custom neural network accelerates have been developed, mostly focused on dense operations [8], [9], [10], [13], [14], [15], [16], [25], [30], [41], [49]. However, edge-centric operations are difficult to implement efficiently on these architectures [3], which are much…”
BackgroundCaffeine: Towards Uniformed Representation and Acceleration for Deep Convolutional Neural Networks
Zhang, C., Sun, G., Fang, Z., Zhou, P., & Cong, J. (2023) · ACM Turing Celebration Conference - China (ACM TURC) · doi:10.1145/3603165.3607390 · Cited by 109
“We implement two levels of parallelism as suggested in [13] for the sake of better hardware utilization and circuit simplicity: 1) parallelism in computing multiple output feature maps; and 2) parallelism in processing multiple input feature maps for each output feature map.”
InfluentialSelf-citationPipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
Li, Y., Yu, M., Li, S., Avestimehr, S., Kim, N. S., & Schwing, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03619 · Cited by 60
“The fraction of time spent on communication is further increased when applying accelerators [16, 7, 38, 52, 48, 49, 44], as they decrease computation time while leaving communication time untouched.”
BackgroundJeon, K., Ryu, J. J., Im, S., Seo, H. K., Eom, T., Ju, H., Yang, M. K., Jeong, D. S., & Kim, G. H. (2024) · Nature Communications · doi:10.1038/s41467-023-44620-1 · Cited by 108
Snowflake: An efficient hardware accelerator for convolutional neural networks
Gokhale, V., Zaidy, A., Chang, A. X. M., & Culurciello, E. (2017) · 2017 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2017.8050809 · Cited by 90
“[9] Caffeine [10] Qiu, et al.”
Radio Frequency Fingerprinting on the Edge
Jian, T., Gong, Y., Zhan, Z., Shi, R., Soltani, N., Wang, Z., Dy, J., Chowdhury, K. R., Wang, Y., & Ioannidis, S. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3064466 · Cited by 109
“Previous works on such hardware implementations study the parallel acceleration of 2D convolutional layers [57], [58]; the 1D convolutions we employ in our architecture (see Section 3) have not received any attention.”
BackgroundMethodAccelerating CNN inference on FPGAs: A Survey
Abdelouahab, K., Pelcat, M., Serot, J., & Berry, F. (2018) · arXiv · doi:10.48550/arxiv.1806.01683 · Cited by 107
“36 improvement in computational throughput vs [39] on the same VX485T device when using 32 floating point arithmetic.”
InfluentialAccelerating CNN Inference on ASICs: A Survey
Moolchandani, D., Kumar, A., & Sarangi, S. R. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2020.101887 · Cited by 110
“…DianNao [21] 2014 NLR - a o , a i 1D array DaDianNao [23] 2014 NLR - a o , a i 1D array ShiDianNao [39] 2015 OS psums a r , a c 2D matrix UCLA [126] 2015 NLR - a o , a i 1D array TPU [63] 2017 WS weights w , l 2D systolic FlexFlow [77] 2017 flexible adaptive flexible 2D matrix Origami [16]…”
BackgroundMethodTowards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA
Shen, J., Huang, Y. R., Wang, Z., Qiao, Y., Wen, M., & Zhang, C. (2018) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3174243.3174257 · Cited by 106
“Since general-purpose CPUs have failed to provide the massive computational parallelism required by modern CNNs, many hardware accelerators (such as GPUs [8], ASICs [6] and FPGAs [2, 14, 21]) have been developed to boost CNN performance.”
BackgroundMethodInfluentialSelf-citationdMazeRunner: Executing Perfectly Nested Loops on Dataflow Accelerators
Dave, S., Kim, Y., Avancha, S., Lee, K., & Shrivastava, A. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3358198 · Cited by 106
“Likewise, [15, 37, 39, 40] considered a batch size of N = 1 images, missing the opportunities for weight reuse.”
BackgroundHardware for machine learning: Challenges and opportunities
Sze, V., Chen, Y.-H., Emer, J., Suleiman, A., & Zhang, Z. (2018) · 2018 IEEE Custom Integrated Circuits Conference (CICC) · doi:10.1109/cicc.2018.8357072 · Cited by 97
NN-Baton: DNN Workload Orchestration and Chiplet Granularity Exploration for Multichip Accelerators
Tan, Z., Cai, H., Dong, R., & Ma, K. (2021) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca52012.2021.00083 · Cited by 104
“In order to reuse data in the on-chip buffer, DNN accelerators require proper loop tiling and loop unrolling that are critical optimization goals in many recent works [6, 15, 32, 41, 50, 71, 76].”
BackgroundHigh-performance video content recognition with long-term recurrent convolutional network for FPGA
Zhang, X., Liu, X., Ramachandran, A., Zhuge, C., Tang, S., Ouyang, P., Cheng, Z., Rupnow, K., & Chen, D. (2017) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.23919/fpl.2017.8056833 · Cited by 104
“Second, FPGAs perform favorably with fixed-point DSP units but not as well when using floating point.”
BackgroundInfluentialNguyen, D. T., Kim, H., & Lee, H.-J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3020569 · Cited by 97
“There are several works that presents a multi-layer processor approach, in which each layer is processed by a dedicated hardware unit [10], [11], [33], [34], to maximize the utilization of computing resources.”
BackgroundJafari, A., Ganesan, A., Thalisetty, C. S. K., Sivasubramanian, V., Oates, T., & Mohsenin, T. (2018) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2018.2848647 · Cited by 103
“[27] analyzed these three tiling methods using the rooftop model to determine what method provides the best throughput in FPGA fabric.”
MethodAn Electro-Photonic System for Accelerating Deep Neural Networks
Demirkiran, C., Eris, F., Wang, G., Elmhurst, J., Moore, N., Harris, N. C., Basumallik, A., Reddi, V. J., Joshi, A., & Bunandar, D. (2023) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3606949 · Cited by 104
Soft errors in DNN accelerators: A comprehensive review
Ibrahim, Y., Wang, H., Liu, J., Wei, J., Chen, L., Rech, P., Adam, K., & Guo, G. (2020) · Microelectronics Reliability · doi:10.1016/j.microrel.2020.113969 · Cited by 102
A Full Featured Configurable Accelerator for Object Detection With YOLO
Pestana, D. G., Miranda, P. R., Lopes, J. D., Duarte, R. P., Véstias, M. P., Neto, H. C., & de Sousa, J. T. (2021) · IEEE Access · doi:10.1109/access.2021.3081818 · Cited by 102
“Zhang et al. [27] were one of the first to employ loop optimisation strategies to accelerate the execution of AlexNet in an FPGA.”
MethodOptimizing CNN-Based Object Detection Algorithms on Embedded FPGA Platforms
Zhao, R., Niu, X., Wu, Y., Luk, W. W., & Liu, Q. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-56258-2_22 · Cited by 102
“[16] use the roofline model and data dependencies analysis to optimise a convolution-only CNN architecture.”
MethodALAMO: FPGA acceleration of deep learning algorithms with a modularized RTL compiler
Ma, Y., Suda, N., Cao, Y., Vrudhula, S., & Seo, J.-S. (2018) · Integration · doi:10.1016/j.vlsi.2017.12.009 · Cited by 93
Achieving Super-Linear Speedup across Multi-FPGA for Real-Time DNN Inference
Jiang, W., Sha, E. H.-M., Zhang, X., Yang, L., Zhuge, Q., Shi, Y., & Hu, J. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3358192 · Cited by 80
“Figure 2 shows the design space exploration of Layer 5 in AlexNet [1] using the model proposed by [14] which is based on roof-line model.”
BackgroundMethodResultInfluentialAutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICs
Xu, P., Zhang, X., Hao, C., Zhao, Y. K., Zhang, Y., Wang, Y., Li, C., Guan, Z., Chen, D., & Lin, Y. (2020) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3373087.3375306 · Cited by 102
“Recently, we have seen intensive studies on DNN accelerators in hardware, which attempt to take advantage of different hardware design styles, such as GPUs, FPGAs, and ASICs, to improve the speed and efficiency of DNN inference and training [10–22].”
BackgroundMethodYonekawa, H., & Nakahara, H. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2017.95 · Cited by 100
TGPA: tile-grained pipeline architecture for low latency CNN inference
Wei, X., Liang, Y. E., Li, X., Yu, C. H., Zhang, P., & Cong, J. (2018) · doi:10.1145/3240765.3240856 · Cited by 100
“We name this design as homogeneous design which has been used in many prior works [2, 15, 23, 27, 29].”
BackgroundMethodSelf-citationPrediction Based Execution on Deep Neural Networks
Song, M., Zhao, J., Hu, Y., Zhang, J., & Li, T. (2018) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2018.00068 · Cited by 100
Hardware Trojan Design on Neural Networks
Clements, J., & Lao, Y. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702493 · Cited by 100
“In the literature, various hardware accelerator designs for neural networks have been proposed [6]–[8], most of which can be generally classified into either temporal or spatial architectures according to the parallelism pattern of the multiply-accumulators (MAC) [2], as seen in Figure 1.”
BackgroundHEIF: Highly Efficient Stochastic Computing-Based Inference Framework for Deep Neural Networks
Li, Z., Li, J., Ren, A., Cai, R., Ding, C., Qian, X., Draper, J. T., Yuan, B., Tang, J., Qiu, Q., et al. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2852752 · Cited by 99
“enormous research interests on field-programmable gate arrays (FPGAs) [12]–[14].”
BackgroundDeepType: On-Device Deep Learning for Input Personalization Service with Minimal Privacy Concern
Xu, M., Qian, F., Mei, Q., Huang, K., & Liu, X. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3287075 · Cited by 74
“• Mobile deep learning Some efforts have been proposed to make deep learning practical on resource-constrained devices, such as model compression [32, 38, 46, 67] and customized hardware architecture support for deep learning [28, 29, 37, 74].”
BackgroundswDNN: A Library for Accelerating Deep Learning Applications on Sunway TaihuLight
Fang, J., Fu, H., Zhao, W., Chen, B., Zheng, W., & Yang, G. (2017) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2017.20 · Cited by 99
“Works in [23], [24] proposed optimized design for the convolutional kernel which achieves considerable performance with single FPGA.”
BackgroundFeng, C., Gu, J., Zhu, H., Ying, Z., Zhao, Z., Pan, D. Z., & Chen, R. T. (2022) · ACS Photonics · doi:10.1021/acsphotonics.2c01188 · Cited by 98
A Survey on Deep-Learning-Based Real-Time SAR Ship Detection
Li, J., Chen, J., Cheng, P., Yu, Z., Yu, L., & Chi, C. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3244616 · Cited by 98
“The ONNX, RunTime, TensorRT, MNN, NCNN, and OpenVino are the inference engine that we can use [136], [137].”
MethodProgramming and Runtime Support to Blaze FPGA Accelerator Deployment at Datacenter Scale
Huang, M., Wu, D., Yu, C. H., Fang, Z., Interlandi, M., Condie, T., & Cong, J. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987569 · Cited by 97
Self-citationAn FPGA-Based Reconfigurable CNN Accelerator for YOLO
Zhang, S., Cao, J., Zhang, Q., Zhang, Q., Zhang, Y., & Wang, Y. (2020) · International Conference on Electronics Technology (ICET) · doi:10.1109/icet49382.2020.9119500 · Cited by 97
“[13] [14] [15] This work”
SpWA: an efficient sparse winograd convolutional neural networks accelerator on FPGAs
Lu, L., & Liang, Y. E. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3195970.3196120 · Cited by 96
Rapuano, E., Meoni, G., Pacini, T., Dinelli, G., Furano, G., Giuffrida, G., & Fanucci, L. (2021) · Remote Sensing · doi:10.3390/rs13081518 · Cited by 96
“Several hardware accelerators proposed in the literature follow this approach [29–31].”
MethodAn FPGA-based accelerator implementation for deep convolutional neural networks
Zhou, Y., & Jiang, J. (2015) · International Conference on Computer Science and Network Technology (ICCSNT) · doi:10.1109/iccsnt.2015.7490869 · Cited by 96
An Overview of Efficient Interconnection Networks for Deep Neural Network Accelerators
Nabavinejad, S. M., Baharloo, M. M., Chen, K.-C. J., Palesi, M., Kogel, T., & Ebrahimi, M. (2020) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2020.3022920 · Cited by 95
Morph: Flexible Acceleration for 3D CNN-Based Video Understanding
Hegde, K., Agrawal, R., Yao, Y., & Fletcher, C. W. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00080 · Cited by 95
“Several prior works study how to search for optimal hardware configurations for FPGA-based accelerators [53], [54], [55], where the entire design can be re-parameterized at configuration time.”
BackgroundTowards Real-Time Object Detection on Embedded Systems
Mao, H., Yao, S., Tang, T., Li, B., Yao, J., & Wang, Y. (2018) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2016.2593643 · Cited by 94
“[14], [15] for CNN and CNN-based vision tasks.”
BackgroundMethodFast and efficient implementation of Convolutional Neural Networks on FPGA
Podili, A., Zhang, C., & Prasanna, V. K. (2017) · IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP) · doi:10.1109/asap.2017.7995253 · Cited by 94
“Many existing FPGA-based CNN implementations[4, 5] cannot perform convolution and pooling concurrently.”
BackgroundResultSelf-citationPCNNA: A Photonic Convolutional Neural Network Accelerator
Mehrabian, A., Al-Kabani, Y., Sorger, V. J., & El-Ghazawi, T. (2018) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc.2018.8618542 · Cited by 94
“This makes photonics an appealing choice for the implementation of CNNs due to their heavy reliance on MAC operation. c (cid:13) 2018 IEEE Previous work on CNN inference hardware has primarily focused on electronic implementations, either in the form of FPGA [6], [7] or ASIC designs [8], [9].”
BackgroundCHARM: C omposing H eterogeneous A ccele R ators for M atrix Multiply on Versal ACAP Architecture
Zhuang, J., Lau, J., Ye, H., Yang, Z., Du, Y., Lo, J. L., Denolf, K., Neuendorffer, S., Jones, A. K., Hu, J., et al. (2023) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3543622.3573210 · Cited by 95
“However, many of the prior works apply a one-size-fits-all monolithic design that cannot efficiently handle layers with huge differences in shapes and sizes (Eyeriss [12, 13], ShiDiannao [14], NPU [15, 16, 17] and others [18, 19, 20, 21]).”
BackgroundSelf-citationA GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks
Li, Y., Liu, Z., Xu, K., Yu, H., & Ren, F. (2018) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3154839 · Cited by 73
“This eliminates any DRAM access latency and dramatically reduces the energy consumption of the system comparing to the existing work relying on off-chip storage [1] [3] [12] [21].”
BackgroundEFLOPS: Algorithm and System Co-Design for a High Performance Distributed Training Platform
Dong, J., Cao, Z., Zhang, T., Ye, J., Wang, S., Feng, F., Li, Z., Liu, X., Song, L., Peng, L., et al. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00056 · Cited by 93
“For instance, various special purpose processors dedicated to accelerating neural networks have been developed in both industry [8], [9], [10], [11], [12] and academia [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23].”
BackgroundBoutros, A., Yazdanshenas, S., & Betz, V. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3242898 · Cited by 93
“For example, Reference [55] presents an analytical design methodology for design space exploration using the roof-line model to find the optimal loop unrolling and tiling parameters for the CONV loops shown in Algorithm 1.”
BackgroundMethodDNNVM: End-to-End Compiler Leveraging Heterogeneous Optimizations on FPGA-Based CNN Accelerators
Xing, Y., Liang, S., Sui, L., Jia, X., Qiu, J., Liu, X., Wang, Y., Shan, Y., & Wang, Y. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2019.2930577 · Cited by 93
“We extend computation to communication (CTC) [40], which describes the computation operations per memory”
BackgroundMethodInfluentialHDLdebugger: Streamlining HDL debugging with Large Language Models
Yao, X., Li, H., Chan, T., Xiao, W., Yuan, M., Huang, Y., Chen, L., & Yu, B. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3735638 · Cited by 96
“• Extensive experiments on the HDL code dataset from Huawei”
AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators
Song, L., Chen, F., Zhuo, Y., Qian, X., Li, H. H., & Chen, Y. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00036 · Cited by 91
“…[24–27], Cambricon [28], Cambricon-x [29], TPU [30,31], ScaleDeep [32], Stripes [33], Neural Cache [123,124], Diffy [125] FPGA FPGA-DCNN [34], Embedded-FPGA-CNN [35], FPGA-Exploration [36], OpenCL-FPGA-CNN [37] [126], Caffeine [38], DeepBurning [39], FPGA-DPCNN [40], TABLA [41],…”
Efficient training and design of photonic neural network through neuroevolution
Zhang, T., Wang, J., Dan, Y., Lanqiu, Y., Dai, J., Han, X., Sun, X., & Xu, K. (2019) · Optics Express · doi:10.1364/oe.27.037150 · Cited by 90
“processing units (GPUs) [14], field-programmable gate arrays (FPGAs) [15] and applicationspecific integrated circuits (ASICs) [16] were invented to accelerate the training and inference process of deep learning.”
MethodUltrafast neuromorphic photonic image processing with a VCSEL neuron
Robertson, J., Kirkland, P., Alanis, J. A., Hejda, M., Bueno, J., Caterina, G. D., & Hurtado, A. (2022) · Scientific Reports · doi:10.1038/s41598-022-08703-1 · Cited by 90
“Electronic processing technologies have grown to alleviate some of this demand [1-3], demonstrating high computational throughput and enabling the development of novel systems for artificial intelligence (AI) [4].”
BackgroundRuntime configurable deep neural networks for energy-accuracy trade-off
Tann, H., Hashemi, S., Bahar, R. I., & Reda, S. (2016) · International Conference on Hardware/Software Codesign and System Synthesis · doi:10.1145/2968456.2968458 · Cited by 90
“propose to remove the less critical neurons in favor of energy reduction [24].”
BackgroundMethodHW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark
Li, C., Yu, Z., Fu, Y., Zhang, Y., Zhao, Y., You, H., Yu, Q., Wang, Y., & Lin, Y. C. (2021) · arXiv · doi:10.48550/arxiv.2103.10584 · Cited by 88
Standing on the Shoulders of Giants: Hardware and Neural Architecture Co-Search With Hot Start
Jiang, W., Yang, L., Dasgupta, S., Hu, J., & Shi, Y. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012863 · Cited by 88
“As the basic operators o in architecture A are conducted in nested loops, the loop optimizations, in particular the loop tiling, are widely studied and used in the design of the computing subsystem in FPGAs [20], [21].”
MethodInfluentialSMAUG: End-to-End Full-Stack Simulation Infrastructure for Deep Learning Workloads
Xi, S. L., Yao, Y., Bhardwaj, K., Whatmough, P. N., Wei, G.-Y., & Brooks, D. M. (2020) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3424669 · Cited by 87
“Other tools focus on exploring dataflows and efficiently map DNN kernels to FPGAS or ASICs [14]–[16], [49]–[51].”
BackgroundA Scalable Near-Memory Architecture for Training Deep Neural Networks on Large In-Memory Datasets
Schuiki, F., Schaffner, M., Gürkaynak, F. K., & Benini, L. (2018) · IEEE Transactions on Computers · doi:10.1109/tc.2018.2876312 · Cited by 87
Liu, S., Fan, H., Niu, X., Ng, H.-C., Chu, Y., & Luk, W. W. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3242900 · Cited by 86
“In particular, FPGA-based accelerators achieve higher performance in terms of execution time compared to CPUs and consume much less power than GPUs while being more flexible and configurable than ASICs [27, 29].”
BackgroundMethodInfluentialFPGA Implementation for CNN-Based Optical Remote Sensing Object Detection
Zhang, N., Wei, X., Chen, H., & Liu, W. (2021) · Electronics · doi:10.3390/electronics10030282 · Cited by 85
“FPGAs and ASICs have the advantages of high performance and energy efficiency [16,19].”
BackgroundInfluentialDesigning efficient accelerator of depthwise separable convolutional neural network on FPGA
Ding, W., Huang, Z., Huang, Z., Tian, L., Wang, H., & Feng, S. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2018.12.008 · Cited by 85
Low-precision Floating-point Arithmetic for High-performance FPGA-based CNN Acceleration
Wu, C. H., Wang, M., Chu, X., Wang, K., & He, L. (2021) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3474597 · Cited by 85
SpWA: An Efficient Sparse Winograd Convolutional Neural Networks Accelerator on FPGAs
Lu, L., & Liang, Y. E. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac.2018.8465842 · Cited by 84
When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies
Yan, Z., Shi, Y., Liao, W., Hashimoto, M., Zhou, X., & Zhuo, C. (2020) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045134 · Cited by 83
“Unlike ResNet56 with residual blocks, VGG net is a more classical deep convolutional neural network with simpler architecture and widely adopted by a variety of perception works [25] and analyzed by a variety of accelerators [22], [26].”
A full-stack search technique for domain optimized deep learning accelerators
Zhang, D., Huda, S., Songhori, E. M., Prabhu, K., Le, Q. V., Goldie, A., & Mirhoseini, A. (2022) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3503222.3507767 · Cited by 87
VIBNN: Hardware Acceleration of Bayesian Neural Networks
Cai, R., Ren, A., Liu, N., Ding, C., Wang, L., Qian, X., Pedram, M., & Wang, Y. (2018) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3173162.3173212 · Cited by 81
“The throughput of a neural network accelerator is mainly determined by computation and communication [57].”
BackgroundTETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
Gao, M., Pu, J., Yang, X., Horowitz, M., & Kozyrakis, C. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3093337.3037702 · Cited by 84
AI Chips: What They Are and Why They Matter
Center for Security and Emerging Technology, Khan, S., & Mann, A. (2020) · doi:10.51593/20190014 · Cited by 47
Xia, M., Huang, Z., Li, T., Wang, H., Chang, V. I. C., Zhu, Y., & Feng, S. (2021) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2021.101991 · Cited by 83
A deep convolutional neural network based on nested residue number system
Nakahara, H., & Sasao, T. (2015) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2015.7293933 · Cited by 82
“Various optimization techniques for the DCNN on the FPGA were proposed: Reduction of multipliers by dynamic reconfiguration [4]; optimization of the memory bandwidth [18], and selection of optimal parameters for the high-level synthesis design [26].”
MethodMoini, S., Alizadeh, B., Emad, M., & Ebrahimpour, R. (2017) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2017.2690919 · Cited by 82
“The results show significant performance to DSP ratio improvement compared to [3] and [5].”
BackgroundMethodResultInfluentialOptimized Compression for Implementing Convolutional Neural Networks on FPGA
Zhang, M., Li, L., Wang, H., Liu, Y., Qin, H., & Zhao, W. (2019) · Electronics · doi:10.3390/electronics8030295 · Cited by 83
Meloni, P., Capotondi, A., Deriu, G., Brian, M., Conti, F., Rossi, D., Raffo, L., & Benini, L. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3284357 · Cited by 83
“In these design cases [45], multiplyand-accumulate modules are smaller and are reused over different cycles to implement filters of different sizes.”
BackgroundFPGA accelerates deep residual learning for image recognition
Li, X., Ding, L., Wang, L., & Cao, F. (2017) · IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) · doi:10.1109/itnec.2017.8284852 · Cited by 81
“More recently, FPGAs have shown promise in efficiently implementing CNNs [5–9].”
SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing
Ren, A., Li, Z., Ding, C., Qiu, Q., Wang, Y., Li, J., Qian, X., & Yuan, B. (2017) · ACM SIGPLAN Notices · doi:10.1145/3093336.3037746 · Cited by 81
Eager pruning: algorithm and architecture support for fast training of deep neural networks
Zhang, J., Chen, X., Song, M., & Li, T. (2019) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3307650.3322263 · Cited by 81
“[21], [22] adopt other data reuse policies but the same problem exists.”
BackgroundMethodLi, J., Un, K.-F., Yu, W.-H., Mak, P., & Martins, R. P. (2021) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2021.3095283 · Cited by 80
Jiang, L., Kim, M., Wen, W., & Wang, D. (2017) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped.2017.8009163 · Cited by 81
“Various accelerators, such as FPGAs [32], [22], ASICs [7], [1], [2], hybrid memory cube (HMC) [17] and ReRAM [3], [28] process-in-memories”
BackgroundInfluentialAutomatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators
Ma, Y., Cao, Y. K., Vrudhula, S. B. K., & Seo, J.-S. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2884972 · Cited by 80
“As mentioned before, most layers in VGG-16 have large CTC ratios that makes them more likely to be computation bounded, thus the throughput improvement of VGG-16 can benefit more from the increase of MAC units than the other three CNNs.”
BackgroundMethodInfluentialAngel-Eye: A Complete Design Flow for Mapping CNN onto Customized Hardware
Guo, K., Sui, L., Qiu, J., Yao, S., Han, S., Wang, Y., & Yang, H. (2016) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2016.129 · Cited by 79
“On the hardware side, an architecture with high efficiency and high flexibility is needed.”
BackgroundFA3C: FPGA-Accelerated Deep Reinforcement Learning
Cho, H., Oh, P., Park, J., Jung, W., & Lee, J. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304058 · Cited by 79
“In other (inference-only) FPGA-based DNN implementations, the basic computation unit is an array of multipliers followed by an adder tree [24, 33] or the systolic array [13, 30].”
BackgroundMethodProcrustes: a Dataflow and Accelerator for Sparse Deep Neural Network Training
Yang, D., Ghasemazar, A., Ren, X., Golub, M., Lemieux, G. G. F., & Lis, M. (2020) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro50266.2020.00064 · Cited by 79
OpenFPGA: An Open-Source Framework for Agile Prototyping Customizable FPGAs
Tang, X., Giacomin, E., Chauviere, B., Alacchi, A., & Gaillardon, P. (2020) · IEEE Micro · doi:10.1109/mm.2020.2995854 · Cited by 78
“Fieldprogrammable gate arrays (FPGAs) are ubiquitous components in modern heterogeneous computing systems, playing an enabling role for programmable accelerators.(1) To maximize their execution efficiency, FPGA fabrics have to be tailored for domain-specific applications so that they can cooperate seamlessly with othe…”
BackgroundMeng, P., Althoff, A., Gautier, Q., & Kastner, R. (2016) · Design, Automation, and Test in Europe · doi:10.3850/9783981537079_0350 · Cited by 78
“FPGAs have demonstrated multiple advantages as a heterogeneous accelerator in many different high performance computing tasks [1]–[3].”
BackgroundAutomatic code generation of convolutional neural networks in FPGA implementation
Liu, Z., Dou, Y., Jiang, J., & Xu, J. (2016) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2016.7929190 · Cited by 79
“Previous study[17] has proved that convolutional and fully-connected layers occupy”
BackgroundVIBNN: Hardware Acceleration of Bayesian Neural Networks
Cai, R., Ren, A., Liu, N., Ding, C., Wang, L., Qian, X., Pedram, M., & Wang, Y. (2018) · ACM SIGPLAN Notices · doi:10.1145/3296957.3173212 · Cited by 79
Huang, M., Luo, J., Ding, C., Wei, Z., Huang, S., & Yu, H. (2023) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2023.3312775 · Cited by 77
“The last challenge associated with Transformer is the bottleneck in memory access [17].”
BackgroundPIXEL: Photonic Neural Network Accelerator
Shiflett, K., Wright, D., Karanth, A., & Louri, A. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00046 · Cited by 77
“…has recently been introduced focusing on increasing ANN computing speed and power efficiency by developing electronic architectures (such as ASIC and FPGA chips) specifically tailored to improve the data computation or storage ability of the accelerator [8], [9], [7], [10], [11], [12], [13], [14].”
BackgroundFlexCNN: An End-to-end Framework for Composing CNN Accelerators on FPGA
Basalama, S., Sohrabizadeh, A., Wang, J., Guo, L., & Cong, J. (2022) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3570928 · Cited by 77
“[59] showed that the CTC ratio of a single convolution layer varies with diferent tiling factors.”
BackgroundSelf-citationPower Line Recognition From Aerial Images With Deep Learning
Yetgin, Ö. E., Benlıgıray, B., & Gerek, Ö. N. (2018) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2018.2883879 · Cited by 77
“Note that it is possible to optimize the method for a lower running time by using lightweight architectures such as MobileNets [51], pruning existing architectures [52], or using specialized hardware [53].”
MethodSLPoW: Secure and Low Latency Proof of Work Protocol for Blockchain in Green IoT Networks
Yazdinejad, A., Srivastava, G., Parizi, R. M., Dehghantanha, A., Karimipour, H., & Karizno, S. R. (2020) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2020-spring48590.2020.9129462 · Cited by 76
“…hash method to increase the security of the IoT networks. ii) We implement SLPoW using FPGA to achieve a high-978-1-7281-5207-3 processing [9], yet computationally and energy efficient PoW protocol which is suitable for IoT constrained networks that which to operate under Green Technology…”
MethodMemristive Circuit Design of Brain-Like Emotional Learning and Generation
Wang, Z., Wang, X., & Zeng, Z. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3090811 · Cited by 79
“In terms of hardware realization, most of the hardware circuits are designed by using complementary metal-oxide semiconductor (CMOS)-based computing elements, such as field-programmable gate array (FPGA) [22], [23].”
BackgroundHardware-Software Codesign of Accurate, Multiplier-free Deep Neural Networks
Tann, H., Hashemi, S., Bahar, R. I., & Reda, S. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062259 · Cited by 63
“In the hardware design domain, while few work have considered different bit-width fixed-point representations in their accelerator design [8, 16, 21], in contrast to the accuracy analysis, no evaluation of hardware designs using dynamic fixed-point is available.”
MethodIncremental On-Device Tiny Machine Learning
Disabato, S., & Roveri, M. (2020) · International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things (AIChallengeIoT) · doi:10.1145/3417313.3429378 · Cited by 76
“Unfortunately, these advantages come at the expense of increased complexity of the design phase together with reduced flexibility [8, 16, 37].”
Background
Metadata
- Authors
- Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J.
- Affiliations
- Peking University (CN)
- University of California, Los Angeles (US)
- Venue
- Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
- Publisher
- ACM
- Published
- 2015-02-22
- Pages
- 161-170
- OpenAlex
- W2094756095
- Cited by
- 2,158
- References
- 17
- Citations by year
- 2026: 78 · 2025: 116 · 2024: 145 · 2023: 162 · 2022: 233 · 2021: 253 · 2020: 258 · 2019: 303 · 2018: 281 · 2017: 221 · 2016: 90 · 2015: 15
- FWCI
- 133.92
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) (publishedVersion)
- Funders
- C-FAR
- NSF China (61202072)
- National High Technology Research and Development Program of China (2012AA010902)
- RFDP (20110001110099)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
- Keywords
- FPGA accelerator, deep convolutional neural networks, roofline model, memory bandwidth, loop tiling, loop transformation, FPGA resource utilization, image recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.