CNP: An FPGA-based processor for Convolutional Networks
The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Abstract
Convolutional networks (ConvNets) are biologically inspired hierarchical architectures that can be trained to perform a variety of detection, recognition and segmentation tasks. ConvNets have a feed-forward architecture consisting of multiple linear convolution filters interspersed with pointwise non-linear squashing functions. This paper presents an efficient implementation of ConvNets on a low-end DSP-oriented field programmable gate array (FPGA). The implementation exploits the inherent parallelism of ConvNets and takes full advantage of multiple hardware multiply accumulate units on the FPGA. The entire system uses a single FPGA with an external memory module, and no extra parts. A network compiler software was implemented, which takes a description of a trained ConvNet and compiles it into a sequence of instructions for the ConvNet Processor (CNP). A ConvNet face detection system was implemented and tested. Face detection on a 512 times 384 frame takes 100 ms (10 frames per second), which corresponds to an average performance of 3.4 times 109connections per second for this 340 million connection network. The design can be used for low-power, lightweight embedded vision systems for micro-UAVs and other small robots.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another option is to use field programable gate arrays (FPGAs), which consume much less energy than GPUs while providing an intermediate computing efficiency between CPUs and GPUs13–17.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [13]
In the citing paper
“Also, CNN can be deployed in medical image classification [114], [115], traffic scenes related classification [116], [117], etc. [118], [119]”
From A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2022) · cites this paper as [116]
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 4×
“The design is specifically geared towards Convolutional Networks [1, 2], but can be used for many similar architectures based on local filter banks and classifiers, such as HMAX [3, 4], and HoG methods [5].”
BackgroundMethodInfluentialSelf-citationHistograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · cited 1×
“The design is specifically geared towards Convolutional Networks [1, 2], but can be used for many similar architectures based on local filter banks and classifiers, such as HMAX [3, 4], and HoG methods [5].”
MethodRapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
GradientBased Learning Applied to Document Recognition
(2009) · Intelligent Signal Processing · doi:10.1109/9780470544976.ch9 · Cited by 1,135 · cited 5×
“This paper presents a self-contained, high performance implementation of Convolutional Networks on a single FPGA.”
BackgroundMethodInfluentialUnsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F. J., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,086 · cited 1×
“ConvNets are used in several commercial and experimental applications, including video surveillance, OCR [2], face/person detection [6, 7], object recognition [8], and robot navigation [9, 10].”
MethodSelf-citationGeneralization and network design strategies
LeCun, Y. (1989) · Cited by 885
Self-citationObject Recognition with Features Inspired by Visual Cortex
Serre, T., Wolf, L., & Poggio, T. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.254 · Cited by 821 · cited 1×
“The design is specifically geared towards Convolutional Networks [1, 2], but can be used for many similar architectures based on local filter banks and classifiers, such as HMAX [3, 4], and HoG methods [5].”
MethodConvolutional face finder: a neural architecture for fast and robust face detection
Garcia, C., & Delakis, M. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.97 · Cited by 547 · cited 3×
“The architecture of the network is quite similar to those described in [6, 7].”
MethodMulticlass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453 · cited 1×
“The design is specifically geared towards Convolutional Networks [1, 2], but can be used for many similar architectures based on local filter banks and classifiers, such as HMAX [3, 4], and HoG methods [5].”
Method(2005) · neural information processing systems · Cited by 446
Self-citation
Research works printed but not linked: 1
- shoup, 1994, parameterized convolution filtering in a field programmable gate array, Selected papers from the Oxford 1993 international workshop on field programmable logic and applications on More FPGAs, 274
Cited by
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084827 · Cited by 5,273
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213
“group [51] achieves 10GOPS for 7x7 kernels, with an architecture that implements entire ConvNets, including pre/postprocessing, and is entirely programmable.”
BackgroundSelf-citationOptimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158
EIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937
ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001139 · Cited by 1,536
“CNNs have also been mapped to FPGAs [11], [15], [16].”
MethodDeep learning in bioinformatics
Min, S., Lee, B., & Yoon, S. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw068 · Cited by 1,465
“Currently, field programmable gate array based processors are under development, and neuromorphic chips modeled from the brain are greatly anticipated as promising technologies [212-214].”
BackgroundReal-Time Motor Fault Detection by 1-D Convolutional Neural Networks
Ince, T., Kiranyaz, S., Eren, L., Askar, M., & Gabbouj, M. (2016) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2582729 · Cited by 1,407
“suitable for FPGA or ASIC implementations [33].”
Going Deeper with Embedded FPGA Platform for Convolutional Neural Network
Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., Song, S., et al. (2016) · Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2847263.2847265 · Cited by 1,252
“To address this problem, many researchers have proposed various CNN acceleration techniques from either computing or memory access aspects [6, 7, 8, 9, 10, 11, 12, 13].”
BackgroundIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080246 · Cited by 1,240
Chi, P., Li, S., Xu, C., Zhang, T., Zhao, J., Liu, Y., Wang, Y., & Xie, Y. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001140 · Cited by 1,231
“There are also many other studies on accelerating NNs on the platforms of GPU [16], [41], [42], FPGA [43], [44], [45] and ASIC [17], [18], [19], [36], [37], [46].”
FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
Umuroglu, Y., Fraser, N. J., Gambardella, G., Blott, M., Leong, P., Jahre, M., & Vissers, K. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021744 · Cited by 1,189
“[7] describe a programmable ConvNet Processor (CNP), which is a RISC vector processor with specific macro-instructions for CNNs including 2D convolutions, 2D spatial pooling, dot product and an elementwise non-linear mapping function.”
BackgroundMethodEIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
ShiDianNao: shifting vision processing closer to the sensor
Du, Z., Fasthuber, R., Chen, T., Ienne, P., Li, L., Luo, T., Feng, X., Chen, Y., & Temam, O. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2750389 · Cited by 933
A survey on deep learning based bearing fault diagnosis
Hoang, D.-T., & Kang, H.-J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.06.078 · Cited by 866
ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.12 · Cited by 775
A generic intelligent bearing fault diagnosis system using compact adaptive 1D CNN classifier
Eren, L., Ince, T., & Kiranyaz, S. (2019) · Journal of Signal Processing Systems · doi:10.1007/s11265-018-1378-3 · Cited by 723
Alwani, M., Chen, H., Ferdman, M., & Milder, P. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783725 · Cited by 662
“In [6], [5], [16], systolic implementations were proposed.”
Cambricon-X: An accelerator for sparse neural networks
Zhang, S., Du, Z., Zhang, L., Lan, H., Liu, S., Li, L., Guo, Q., Chen, T., & Chen, Y. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783723 · Cited by 603
Chi, P., Li, S., Xu, C., Zhang, T., Zhao, J., Liu, Y., Wang, Y., & Xie, Y. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.13 · Cited by 573
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
“The computing time can be accelerated by using graphics processing units (GPUs) to implement the ANNs (see Fig.”
Edge Machine Learning for AI-Enabled IoT Devices: A Review
Merenda, M., Porcaro, C., & Iero, D. (2020) · Sensors · doi:10.3390/s20092533 · Cited by 523
A Configurable Cloud-Scale DNN Processor for Real-Time AI
Fowers, J., Ovtcharov, K., Papamichael, M., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., Adams, L., Ghandi, M., et al. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00012 · Cited by 520
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review
Shawahna, A., Sait, S. M., & El-Maleh, A. (2019) · IEEE Access · doi:10.1109/access.2018.2890150 · Cited by 496
“CNP represents FMs and weights using 16-bit (Q8.”
BackgroundMethodInfluentialFINN- R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
Blott, M., Preußer, T. B., Fraser, N. J., Gambardella, G., O’brien, K., Umuroglu, Y., Leeser, M., & Vissers, K. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3242897 · Cited by 446
“hitecture [4, 49, 60], consisting of one processing engine per network layer where inputs are streamed through the dataflow architecture and all layers are computed in parallel; 3) a vector processor [17] with instructions specific to accelerating the primitive operations of convolutions. Recent graphics processing un…”
BackgroundMethodAutomated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs
Wei, X., Yu, C. H., Zhang, P., Chen, Y., Wang, Y., Hu, H., Liang, Y., & Cong, J. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062207 · Cited by 426
“The studies explore parallelism opportunities in input feature maps [2] and convolution kernels [1, 4]; while the work in [3] chooses to parallelize output feature maps.”
BackgroundPUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltin, M., Williams, R. S., Faraboschi, P., Hwu, W.-M. W., Strachan, J. P., Roy, K., et al. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304049 · Cited by 424
“rs specialized for a particular field [14, 79, 101, 114], to accelerators that are fully programmable via an ISA [61, 71, 72, 105, 119]. FPGAs have also been popular targets for building accelerators [37, 45, 74, 86, 96, 97, 110, 117]. All these works remain in the digital domain, while PUMA leverages hybrid digital-a…”
BackgroundMinimizing Computation in Convolutional Neural Networks
Cong, J., & Xiao, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11179-7_36 · Cited by 372
“, GPUs and field-programmable gate arrays (FPGAs) [3,5,9].”
BackgroundNeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369
“It can also perform on-the-fly subsampling (spatial pooling), and simple dot-products (linear classifiers) [10] • the middle row PTs contain general purpose operators, such as squaring and dividing for divisive normalization, and other mathematical operators • the bottom row PTs implement non-linear…”
BackgroundMethodSelf-citationA Survey of Neuromorphic Computing and Neural Networks in Hardware
Schuman, C. D., Potok, T. E., Patton, R. M., Birdwell, J. D., Dean, M. E., Rose, G. S., & Plank, J. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06963 · Cited by 352
“recognition or detection [582], [1067], [1204], [1284], [1885],”
SqueezeNext: Hardware-Aware Neural Network Design
Gholami, A., Kwon, K., Wu, B., Tai, Z., Yue, X., Jin, P., Zhao, S., & Keutzer, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00215 · Cited by 350
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343
“Results for the 200 MHz CNP (column “CNP” in Table 3) were taken from [9].”
BackgroundMethodInfluentialNeurocube: a programmable digital neuromorphic architecture with high-density 3D memory
Kim, D., Kung, J., Chai, S., Yalamanchili, S., & Mukhopadhyay, S. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001178 · Cited by 342
“While there are architectures that attempt to achieve performance and scalability by reducing the processing units [3]–[5], [7], [8], the overall system performance is often limited due to insufficient memory bandwidth and network latency.”
BackgroundAccelerating Binarized Neural Networks: Comparison of FPGA, CPU, GPU, and ASIC
Nurvitadhi, E., Sheffield, D., Sim, J., Mishra, A., Venkatesh, G., & Marr, D. (2016) · 2016 International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2016.7929192 · Cited by 342
FlexFlow: A Flexible Dataflow Accelerator Architecture for Convolutional Neural Networks
Lu, W., Yan, G., Li, J., Gong, S., Han, Y., & Li, X. (2017) · 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2017.29 · Cited by 333
“Work [8] presents a self-contained FPGA implementation of CNN .”
BackgroundMethodCaffeine: towards uniformed representation and acceleration for deep convolutional neural networks
Zhang, C., Fang, Z., Zhou, P., Pan, P., & Cong, J. (2016) · Proceedings of the 35th International Conference on Computer-Aided Design · doi:10.1145/2966986.2967011 · Cited by 299
“Most prior FPGA acceleration studies on CNN [13, 15–21] mainly focus on the convolution layer in CNN, since it is computation-bound and is the most timing-consuming layer.”
BackgroundA high performance FPGA-based accelerator for large-scale convolutional neural networks
Li, H., Fan, X., Jiao, L., Cao, W., Zhou, X., & Wang, L. (2016) · 2016 26th International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2016.7577308 · Cited by 299
“Hence, general purpose processors are not so efficient while a number of CNN implementations based on GPU [1], FPGA [7, 8, 9, 10, 11, 12, 13, 14, 15] even ASIC [16, 17, 18, 19] have been proposed.”
BackgroundFP-BNN: Binarized neural network on FPGA
Liang, S., Yin, S., Liu, L., Luk, W., & Wei, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.09.046 · Cited by 298
ShiDianNao: shifting vision processing closer to the sensor
Du, Z., Fasthuber, R., Chen, T., Ienne, P., Li, L., Luo, T., Feng, X., Chen, Y., & Temam, O. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/2872887.2750389 · Cited by 148
Maximizing CNN Accelerator Efficiency Through Resource Partitioning
Shen, Y., Ferdman, M., & Milder, P. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080221 · Cited by 284
“For example, the 2D-convolvers used in [3, 10, 12, 22] must be provisioned for the largest filter across layers; they will necessarily be underutilized when computing layers with smaller filters.”
MethodCaffeine: Toward Uniformed Representation and Acceleration for Deep Convolutional Neural Networks
Zhang, C., Sun, G., Fang, Z., Zhou, P., Pan, P., & Cong, J. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2785257 · Cited by 270
“Most prior FPGA acceleration studies on CNN [13], [16]–[22], [26] mainly focus on the convolution layer in CNN, since it is computation-bound and is the most timing-consuming layer.”
BackgroundfpgaConvNet: A Framework for Mapping Convolutional Neural Networks on FPGAs
Venieris, S. I., & Bouganis, C.-S. (2016) · 2016 IEEE 24th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2016.22 · Cited by 267
“Nevertheless, systolic implementations suffer from complex routing logic and can support convolutions only up to the maximum implemented kernel size, e.g. 7x7 and 10x10 convolutions in CNP and nn-X respectively.”
BackgroundInfluentialBearing fault detection by one-dimensional convolutional neural networks
Eren, L. (2017) · Mathematical Problems in Engineering · doi:10.1155/2017/8617315 · Cited by 250
Evaluating Fast Algorithms for Convolutional Neural Networks on FPGAs
Lu, L., Liang, Y., Xiao, Q., & Yan, S. (2017) · 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.64 · Cited by 248
Cambricon: an instruction set architecture for neural networks
Liu, S., Du, Z., Tao, J., Han, D., Luo, T., Xie, Y., Chen, Y., & Chen, T. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001179 · Cited by 243
Deep Learning on FPGAs: Past, Present, and Future
Lacey, G., Taylor, G. W., & Areibi, S. M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04283 · Cited by 148
“In this paper, Zhang et al. show their work to outperform most of the main strong competitors in this field, including [14, 15, 23, 32, 33].”
BackgroundA Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification
Rokh, B., Azarpeyvand, A., & Khanteymoori, A. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3623402 · Cited by 232
“Later, researchers such as Song Han et al. [17], Matthieu Courbariaux et al. [21], Mohammad Rastegari et al. [22], and Shuchang Zhou et al. [23], concluded that compression and software optimization of DNNs can be more effective before touching hardware.”
BackgroundHardware accelerated convolutional neural networks for synthetic vision systems
Farabet, C., Martini, B., Akselrod, P., Talay, S., LeCun, Y., & Culurciello, E. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537908 · Cited by 231
“The FPGA was connected to external QDR-SRAM memory in a custom designed printed circuit board [8].”
BackgroundMethodInfluentialSelf-citationAn OpenCL™ Deep Learning Accelerator on Arria 10
Aydonat, U., O'Connell, S., Capalija, D., Ling, A. C., & Chiu, G. R. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021738 · Cited by 229
“More recently, FPGAs have shown promise in efficiently implementing CNNs [21, 16, 13, 20, 11, 2, 12, 14, 4].”
BackgroundMethodEnergy-Efficient CNN Implementation on a Deeply Pipelined FPGA Cluster
Zhang, C., Wu, D., Sun, J., Sun, G., Luo, G., & Cong, J. (2016) · Proceedings of the 2016 International Symposium on Low Power Electronics and Design · doi:10.1145/2934583.2934644 · Cited by 227
“Previous work mostly focused on single-Permission board implementation [8, 10, 11, 12, 13].”
Capra, M., Bussolino, B., Marchisio, A., Masera, G., Martina, M., & Shafique, M. (2020) · IEEE Access · doi:10.1109/access.2020.3039858 · Cited by 222
“In CNP [260] for example, in order to accommodate a large number of DSPs on a Virtex4 SX35 FPGA platform, the authors designed an interface with an external memory capable of performing 8 read/write operations.”
MethodVoiceHD: Hyperdimensional Computing for Efficient Speech Recognition
Imani, M., Kong, D., Rahimi, A., & Rosing, T. (2017) · 2017 IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc.2017.8123650 · Cited by 216
“Prior work tried to accelerate NNs on GPUs [19], [20], FPGAs [21], [22], and ASICs [23], [24].”
BackgroundCambricon: An Instruction Set Architecture for Neural Networks
Liu, S., Du, Z., Tao, J., Han, D., Luo, T., Xie, Y., Chen, Y., & Chen, T. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.42 · Cited by 121
Design space exploration of FPGA-based Deep Convolutional Neural Networks
Motamedi, M., Gysel, P., Akella, V., & Ghiasi, S. (2016) · 2016 21st Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2016.7428073 · Cited by 206
“ICCD2013[8] FPL2009[6] PACT2010[1] ISCA2010[2] ISFPGA2015[10] Proposed Sol.”
Analog architectures for neural network acceleration based on non-volatile memory
Xiao, T. P., Bennett, C. H., Feinberg, B., Agarwal, S., & Marinella, M. J. (2020) · Applied Physics Reviews · doi:10.1063/1.5143815 · Cited by 193
A systematic literature review on hardware implementation of artificial intelligence algorithms
Talib, M. A., Majzoub, S., Nasir, Q., & Jamal, D. (2021) · The Journal of Supercomputing · doi:10.1007/s11227-020-03325-8 · Cited by 188
“CNNs are feed-forward architecture that consists of multiple linear convolution filters, interspersed with point-wise nonlinear squashing functions [7].”
BackgroundOrigami: A Convolutional Network Accelerator
Cavigelli, L., Gschwend, D., Mayer, C., Willi, S., Muheim, B., & Benini, L. (2015) · Proceedings of the 25th edition on Great Lakes Symposium on VLSI · doi:10.1145/2742060.2743766 · Cited by 156
“2 Fixed-Point Analysis Previous work is not conclusive on the required precision for ConvNets, 16 and 18 bit are the most common values [14], [15], [17], [18].”
BackgroundInfluentialEfficient Hardware Architectures for Accelerating Deep Neural Networks: Survey
Pudi, D., Boppu, S., Manikandan, M. S., & Cenkeramaddi, L. R. (2022) · IEEE Access · doi:10.1109/access.2022.3229767 · Cited by 180
“Farabet et al. [85] presented ConvNet Processor (CNP): an FPGA-based accelerator to implement the CNNs. CNP uses dedicated hardware convolver for the data processing and also uses soft-processor for controlling.”
BackgroundMethodInfluentialOptimizing CNN Model Inference on CPUsPreprint
Liu, Y., Wang, Y., Yu, R., Li, M., Sharma, V., & Wang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02697 · Cited by 181
“…demonstrates more and more power in the real-world applications, there is a significant amount of effort being made to accelerate the deep learning workloads on all kinds of hardware ranging from CPUs [20, 23, 40, 46], GPUs [8, 10], FPGAs [15, 17, 45], to special-purpose accelerators [9, 26].”
BackgroundA dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · ACM SIGARCH Computer Architecture News · doi:10.1145/1816038.1815993 · Cited by 151
Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC
Nurvitadhi, E., Sim, J., Sheffield, D., Mishra, A., Krishnan, S., & Marr, D. (2016) · 2016 26th International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2016.7577314 · Cited by 175
“, [10][11]), but they are not for RNN and its variants.”
BackgroundNeurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory
Kim, D., Kung, J., Chai, S., Yalamanchili, S., & Mukhopadhyay, S. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.41 · Cited by 171
OPU: An FPGA-Based Overlay Processor for Convolutional Neural Networks
Yu, Y., Wu, C. H., Zhao, T., Wang, K., & He, L. (2019) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2939726 · Cited by 172
“[1] used FPGA as a vector-based arithmetic unit, and implemented”
BackgroundMethodOrigami: A 803-GOp/s/W Convolutional Network Accelerator
Cavigelli, L., & Benini, L. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2592330 · Cited by 169
“It also features a compiler, converting network implementations with Torch directly to CNP instructions.”
BackgroundInfluentialModeling Deep Learning Accelerator Enabled GPUs
Raihan, A., Goli, N., & Aamodt, T. M. (2019) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2019.00016 · Cited by 163
ConfuciuX: Autonomous Hardware Resource Assignment for DNN Accelerators using Reinforcement Learning
Kao, S.-C., Jeong, G., & Krishna, T. (2020) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro50266.2020.00058 · Cited by 150
“Naturally, over-provisioning and under-utilization may happen for some of the layers during deployment since they favor different HW resource configuration [11], [21], [22], [63].”
BackgroundLarge-Scale FPGA-Based Convolutional Networks
Farabet, C., Lecun, Y., Kavukcuoglu, K., Martini, B., Akselrod, P., Talay, S., & Culurciello, E. (2011) · Scaling up Machine Learning · doi:10.1017/cbo9781139042918.020 · Cited by 121
“The pipelined implementation of this 2D convolver (as described in Farabet et al. (2009)), computes Equation 8 at every clock cycle. y1,i,j = x2,i,j + K−1∑ m=0 K−1∑ n=0 x1,i+m,j+nw1,m,n (8) In equation 8 x1,i,j is a value in the input plane, w1,m,n is a value in a K×K convolution kernel, x2,i,j is…”
BackgroundMethodSelf-citationThroughput-Optimized FPGA Accelerator for Deep Convolutional Neural Networks
Liu, Z., Dou, Y., Jiang, J., Xu, J., Li, S., Zhou, Y., & Xu, Y. (2017) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3079758 · Cited by 116
“However, some previous works only considered small CNNs for simple tasks [Chen et al. 2014; Farabet et al. 2009; Chakradhar et al. 2010].”
BackgroundDarbon, J., Langlois, G. P., & Meng, T. (2020) · Research in the Mathematical Sciences · doi:10.1007/s40687-020-00215-6 · Cited by 135
“In addition, there has been many initiatives to create new hardware for neural networks that yields extremely efficient (in terms of speed, latency, throughput or energy) implementations: for instance, [51, 52, 53] propose efficient neural network implementations using field-programmable gate array, [8] optimizes neur…”
BackgroundLight-OPU: An FPGA-based Overlay Processor for Lightweight Convolutional Neural Networks
Yu, Y., Zhao, T., Wang, K., & He, L. (2020) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3373087.3375311 · Cited by 130
“Conventional convolutional neural network (CNN) acceleration on FPGA has drawn much attention in recent years [2, 4, 7, 16, 18, 21, 24, 28, 34, 37–39].”
BackgroundNeurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes
Azarkhish, E., Rossi, D., Loi, I., & Benini, L. (2017) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2017.2752706 · Cited by 129
“s in large data centers. A diverse range of ConvNet implementations exist today ranging from standard software libraries running on generalpurpose platforms [18] [19] [20] to application-specific FPGA [21] [22] [23] [24] and ASIC implementations [25] [26] [27] [28]. High-performance supercomputers such as NVIDIA DGX-1…”
BackgroundMaximizing CNN Accelerator Efficiency Through Resource Partitioning
Shen, Y., Ferdman, M., & Milder, P. A. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080221 · Cited by 124
An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors
Camuñas-Mesa, L. A., Zamarreño-Ramos, C., Linares-Barranco, A., Acosta, A. J., Serrano‐Gotarredona, T., & Linares-Barranco, B. (2011) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2011.2167409 · Cited by 122
“Table III summarizes recognition delays of three example systems that perform face recognition with ConvNets, implemented either with GPUs [33] or FPGAs [30], [32], [34], [35], when using VGA-like size input images.”
MethodEfficient FPGA Acceleration of Convolutional Neural Networks Using Logical-3D Compute Array
Rahman, A., Lee, J., & Choi, K. (2016) · Design, Automation, and Test in Europe · doi:10.3850/9783981537079_0833 · Cited by 120
“Our solution is a novel architecture called Input-recycling Convolutional Array of Neurons (ICAN), which differs from previous solutions by parallelizing along three carefully chosen dimensions that mimics the convolution operation used in DNNs.”
BackgroundMethodInfluentialHolyLight: A Nanophotonic Accelerator for Deep Learning in Data Centers
Liu, W., Liu, W., Ye, Y., Lou, Q., Xie, Y., & Jiang, L. (2019) · Design, Automation, and Test in Europe · doi:10.23919/date.2019.8715195 · Cited by 119
“LeNet-5, CNP, SCNN and MCDNN were trained with MNIST to identify simple handwritten digits, while AlexNet and ResNet-18 were trained with ImageNet to recognize complex objects.”
MethodAn FPGA-Based CNN Accelerator Integrating Depthwise Separable Convolution
Liu, B., Zou, D., Feng, L., Feng, S., Fu, P., & Li, J. (2019) · Electronics · doi:10.3390/electronics8030281 · Cited by 117
Escher: A CNN Accelerator with Flexible Buffering to Minimize Off-Chip Transfer
Shen, Y., Ferdman, M., & Milder, P. (2017) · 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.47 · Cited by 106
Accelerating CNN inference on FPGAs: A Survey
Abdelouahab, K., Pelcat, M., Serot, J., & Berry, F. (2018) · arXiv · doi:10.48550/arxiv.1806.01683 · Cited by 107
“It can only support convolutions with a kernel size K that is smaller than a given maximum size K m (i.e support only convolutions such K ≤ K m where , for instance, K m = 7 in [73] and K m = 10 in [76]).”
InfluentialAccelerating CNN Inference on ASICs: A Survey
Moolchandani, D., Kumar, A., & Sarangi, S. R. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2020.101887 · Cited by 108
“Parallelism Type Architecture Type Intra-output Inter-output Inter-Kernel Intra-Kernel CNP [42] 2D systolic DC-CNN [17] 1D array MAPLE [15] 2D systolic nn-x [44] 1D systolic [61] 1D systolic C-Brain [113] 1D array FlexFlow* [77] 2D matrix Origami [16] 1D array SCNN* [94] 2D systolic YodaNN* [11] 1D…”
Yonekawa, H., & Nakahara, H. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2017.95 · Cited by 100
An Energy-Efficient Architecture for Binary Weight Convolutional Neural Networks
Wang, Y., Lin, J. T., & Wang, Z. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2767624 · Cited by 98
“Previous works [10], [22]–[24] used the most common values of 16 or 12 b for both storage and computation precision of feature maps (activations).”
MethodTestability and Dependability of AI Hardware: Survey, Trends, Challenges, and Perspectives
Su, F., Liu, C., & Stratigopoulos, H. (2023) · IEEE Design and Test · doi:10.1109/mdat.2023.3241116 · Cited by 97
An FPGA-based accelerator implementation for deep convolutional neural networks
Zhou, Y., & Jiang, J. (2015) · International Conference on Computer Science and Network Technology (ICCSNT) · doi:10.1109/iccsnt.2015.7490869 · Cited by 96
Towards Real-Time Object Detection on Embedded Systems
Mao, H., Yao, S., Tang, T., Li, B., Yao, J., & Wang, Y. (2018) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2016.2593643 · Cited by 94
“[14], [15] for CNN and CNN-based vision tasks.”
MethodWang, P., Ji, Y., Hong, C., Lyu, Y., Wang, D., & Xie, Y. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3195970.3196116 · Cited by 94
“Ac-celerators based on conventional hardware platforms [7, 4, 10] still need considerable hardware investment and high power budget for large-scale DNN deployment.”
BackgroundCompiling KB-sized machine learning models to tiny IoT devices
Gopinath, S., Ghanathe, N., Seshadri, V., & Sharma, R. (2019) · Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/3314221.3314597 · Cited by 92
“There have been several efforts to explore acceleration of ML inference with MB/GB sized models on FPGAs [10, 21, 22, 25, 80].”
BackgroundOn Vectorization of Deep Convolutional Neural Networks for Vision Tasks
Ren, J., & Xu, L. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9488 · Cited by 76
Runtime configurable deep neural networks for energy-accuracy trade-off
Tann, H., Hashemi, S., Bahar, R. I., & Reda, S. (2016) · International Conference on Hardware/Software Codesign and System Synthesis · doi:10.1145/2968456.2968458 · Cited by 89
“[8] propose the use of one hardware convolutional operator for implementing the filtering computation while the rest of the computation is done in software.”
BackgroundMethodDarbon, J., & Meng, T. (2020) · Journal of Computational Physics · doi:10.1016/j.jcp.2020.109907 · Cited by 85
Enhanced bearing fault detection using multichannel, multilevel 1D CNN classifier
Ozcan, I. H., Devecioğlu, Ö. C., İnce, T., Eren, L., & Aşkar, M. (2021) · Electrical Engineering · doi:10.1007/s00202-021-01309-2 · Cited by 82
A deep convolutional neural network based on nested residue number system
Nakahara, H., & Sasao, T. (2015) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2015.7293933 · Cited by 82
“The basic 2D convolutional circuits were reported in [10], [19].”
MethodOptimized Compression for Implementing Convolutional Neural Networks on FPGA
Zhang, M., Li, L., Wang, H., Liu, Y., Qin, H., & Zhao, W. (2019) · Electronics · doi:10.3390/electronics8030295 · Cited by 83
Jiang, L., Kim, M., Wen, W., & Wang, D. (2017) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped.2017.8009163 · Cited by 80
“[8] C. Farabet,et al., “CNP: An FPGA-based processor for Convolutional Networks,” in FPL, 2009.”
MethodInfluentialAutomatic code generation of convolutional neural networks in FPGA implementation
Liu, Z., Dou, Y., Jiang, J., & Xu, J. (2016) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2016.7929190 · Cited by 78
“Some of previous works only considered small CNN models for simple tasks[13][14][15][16].”
BackgroundLi, S., Luo, Y., Sun, K., Yadav, N., & Choi, K. K. (2020) · IEEE Access · doi:10.1109/access.2020.3000009 · Cited by 75
“Farabet et al. [19] placed some data processing tasks on DSP to speed up the processing.”
BackgroundDeepX: Deep Learning Accelerator for Restricted Boltzmann Machine Artificial Neural Networks
Kim, L.-W. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2665555 · Cited by 75
“Reference [16] implemented an efficient FPGA accelerator based on a low-end DSP for CNNs.”
MethodPUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning InferencePreprint
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltín, M., Williams, R. S., Faraboschi, P., Hwu, W., Strachan, J. P., Roy, K., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10351 · Cited by 73
“rs specialized for a particular field [14, 79, 101, 114], to accelerators that are fully programmable via an ISA [61, 71, 72, 105, 119]. FPGAs have also been popular targets for building accelerators [37, 45, 74, 86, 96, 97, 110, 117]. All these works remain in the digital domain, while PUMA leverages hybrid digital-a…”
BackgroundAn In-Memory VLSI Architecture for Convolutional Neural Networks
Kang, M., Lim, S., Gonugondla, S. K., & Shanbhag, N. R. (2018) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2018.2829522 · Cited by 72
“3(b)) accumulates the partial sums computed by Nbank such banks if needed, and then performs the sigmoid function, which is implemented digitally using a piece-wise linear approximation composed of three additions and two shifts [30].”
MethodData and Hardware Efficient Design for Convolutional Neural Network
Lin, Y.-J., & Chang, T. (2017) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2017.2759803 · Cited by 72
“This first design, CNP [38], can execute all networks but face the difficulty of scalability on higher network performance.”
BackgroundMethodScaling Binarized Neural Networks on Reconfigurable Logic
Fraser, N. J., Umuroglu, Y., Gambardella, G., Blott, M., Leong, P. H. W., Jahre, M., & Vissers, K. (2017) · doi:10.1145/3029580.3029586 · Cited by 70
Exploring the design space of specialized multicore neural processors
Taha, T. M., Hasan, R., Yakopcic, C., & McLean, M. R. (2013) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2013.6707074 · Cited by 69
Metadata
- Authors
- Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y.
- Affiliations
- Courant Institute of Mathematical Sciences (US)
- ORCID
- LeCun, Yann
- Venue
- 2009 International Conference on Field Programmable Logic and Applications
- Publisher
- IEEE
- Published
- 2009-08
- Pages
- 32-37
- OpenAlex
- W1990315422
- Cited by
- 367
- References
- 17
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 12 · 2023: 13 · 2022: 19 · 2021: 23 · 2020: 36 · 2019: 46 · 2018: 55 · 2017: 54 · 2016: 51 · 2015: 15 · 2014: 2 · 2013: 4 · 2012: 3
- FWCI
- 6.26
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Field Programmable Logic and Applications (FPL) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
- Keywords
- convolutional neural networks, FPGA implementation, multiply-accumulate unit, face detection, embedded vision, low-power electronics, micro UAV, hardware acceleration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.