A dynamically configurable coprocessor for convolutional neural networks
The DOI matches its Crossref record; 34 of the 36 references it cites were found in a registry or an index.
Abstract
Convolutional neural networks (CNN) applications range from recognition and reasoning (such as handwriting recognition, facial expression recognition and video surveillance) to intelligent text applications such as semantic text analysis and natural language processing applications. Two key observations drive the design of a new architecture for CNN. First, CNN workloads exhibit a widely varying mix of three types of parallelism: parallelism within a convolution operation, intra-output parallelism where multiple input sources (features) are combined to create a single output, and inter-output parallelism where multiple, independent outputs (features) are computed simultaneously. Workloads differ significantly across different CNN applications, and across different layers of a CNN. Second, the number of processing elements in an architecture continues to scale (as per Moore's law) much faster than off-chip memory bandwidth (or pin-count) of chips. Based on these two observations, we show that for a given number of processing elements and off-chip memory bandwidth, a new CNN hardware architecture that dynamically configures the hardware on-the-fly to match the specific mix of parallelism in a given workload gives the best throughput performance. Our CNN compiler automatically translates high abstraction network specification into a parallel microprogram (a sequence of low-level VLIW instructions) that is mapped, scheduled and executed by the coprocessor. Compared to a 2.3 GHz quad-core, dual socket Intel Xeon, 1.35 GHz C870 GPU, and a 200 MHz FPGA implementation, our 120 MHz dynamically configurable architecture is 4x to 8x faster. This is the first CNN architecture to achieve real-time video stream processing (25 to 30 frames per second) on a wide range of object detection and recognition tasks.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another option is to use field programable gate arrays (FPGAs), which consume much less energy than GPUs while providing an intermediate computing efficiency between CPUs and GPUs13–17.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [16]
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Neural Networks And Learning Machines
Haykin, S. (2010) · Cited by 7,534
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 1×
“Feed-forward, multi-layer artificial neural networks like the Convolutional Neural Networks (CNN) [2][11][13][16][17][20][23] have found increasing use in several new applications because they have the potential to process vast amounts of labeled data to automatically learn and extract complex…”
BackgroundFace recognition: a convolutional neural-network approach
Lawrence, S., Giles, C. L., Tsoi, A. C., & Back, A. D. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.554195 · Cited by 3,092
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Object Recognition with Features Inspired by Visual Cortex
Serre, T., Wolf, L., & Poggio, T. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.254 · Cited by 821
Large-scale deep unsupervised learning using graphics processors
Raina, R., Madhavan, A., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553486 · Cited by 742 · cited 2×
“…to match the exact computational workload of any layer of a CNN. Note that by replacing the convolver primitive with a different functional unit, we can easily implement different feed-forward neural networks and classifiers such as HMAX [17][27], Deep Belief Networks [29][30] and HoG methods [28].”
MethodMulticlass Object Recognition with Sparse, Localized Features
Mutch, J., & Lowe, D. G. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.200 · Cited by 453
Fast support vector machine training and classification on graphics processors
Catanzaro, B., Sundaram, N., & Keutzer, K. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390170 · Cited by 372
CNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367 · cited 6×
“Results for the 200 MHz CNP (column “CNP” in Table 3) were taken from [9].”
BackgroundMethodInfluentialFPGA Implementations of Neural Networks
(2006) · Springer eBooks · doi:10.1007/0-387-28487-7 · Cited by 320
Learning long‐range vision for autonomous off‐road driving
Hadsell, R., Sermanet, P., Ben, J., Erkan, A., Scoffier, M., Kavukcuoglu, K., Muller, U., & LeCun, Y. (2009) · Journal of Field Robotics · doi:10.1002/rob.20276 · Cited by 314
Neural networks in finance : gaining predictive edge in the market
McNelis, P. D. (2005) · Cited by 228
Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial EEG
Mirowski, P. W., LeCun, Y., Madhavan, D., & Kuzniecky, R. (2008) · 2008 IEEE Workshop on Machine Learning for Signal Processing · doi:10.1109/mlsp.2008.4685487 · Cited by 169
The Impact of Arithmetic Representation on Implementing MLP-BP on FPGAs: A Study
Savich, A. W., Moussa, M. A., & Areibi, S. M. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.883002 · Cited by 165
Asymptotically fast factorization of integers
Dixon, J. D. (1981) · Mathematics of Computation · doi:10.1090/s0025-5718-1981-0595059-1 · Cited by 136
Artificial Neural Networks in Biomedicine
(2000) · Perspectives in neural computing · doi:10.1007/978-1-4471-0487-2 · Cited by 106
Face Detection Using GPU-Based Convolutional Neural Networks
Nasse, F., Thurau, C., & Fink, G. A. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03767-2_10 · Cited by 71
Speech, Audio, Image and Biomedical Signal Processing using Neural Networks
Prasad, B., & Prasanna, S. R. M. (2007) · Studies in computational intelligence · doi:10.1007/978-3-540-75398-8 · Cited by 62
A Multiwindow Partial Buffering Scheme for FPGA-Based 2-D Convolvers
Zhang, H., Xia, M., & Hu, G. (2007) · IEEE Transactions on Circuits and Systems II Analog and Digital Signal Processing · doi:10.1109/tcsii.2006.886898 · Cited by 60
A 40GOPS 250mW massively parallel processor based on matrix architecture
Nakajima, M., Noda, H., DOSAKA, K., Nakata, K., Higashida, M., Yamamoto, O., Mizumoto, K., Kondo, H., Shimazu, Y., Arimoto, K., et al. (2006) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc.2006.1696216 · Cited by 52
FPGA Implementation of a Pipelined On-Line Backpropagation
Gironés, R. G., Gironés, R. G., Colom, R. J., Boluda, J. C., Boluda, J. C., & Sebastià-Cortés, A. (2005) · The Journal of VLSI Signal Processing Systems for Signal Image and Video Technology · doi:10.1007/s11265-005-4961-3 · Cited by 47
McNelis, P. D. (2004) · Academic Press, Inc. eBooks · Cited by 44
Area-efficient 2-D shift-variant convolvers for FPGA-based digital image processing
Cardells-Tormo, F., & Molinet, P. (2006) · IEEE Transactions on Circuits and Systems II Analog and Digital Signal Processing · doi:10.1109/tcsii.2005.857091 · Cited by 42
Area-efficient 2-D shift-variant convolvers for FPGA-based digital image processing
Cardells-Tormo, F., Molinet, P., Sempere-Agullo, J., Baldez, L., & Bautista-Palacios, M. (2005) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2005.1515789 · Cited by 38
A multirange architecture for collision‐free off‐road robot navigation
Sermanet, P., Hadsell, R. T., Scoffier, M., Grimes, M. K., Ben, J., Erkan, A., Crudele, C., Miller, U., & LeCun, Y. (2008) · Journal of Field Robotics · doi:10.1002/rob.20270 · Cited by 35
Design and implementation of a 2D convolution core for video applications on FPGAs
Benkrid, K., & Belkacemi, S. (2004) · International Workshop on Digital and Computational Video (DCV) · doi:10.1109/dcv.2002.1218747 · Cited by 25
USING EMBEDDED PROCESSORS IN HARDWARE MODELS OF ARTIFICIAL NEURAL NETWORKS
Wolf, D. F., Romero, R. A. F., & Marques, E. (2001) · Cited by 21
Korekado, K., Morie, T., Nomura, O., Nakano, T., Matsugu, M., & Iwata, A. (2005) · Symposium on VLSI Circuits · doi:10.1109/vlsic.2005.1469371 · Cited by 19
Projection-Field-Type VLSI Convolutional Neural Networks Using Merged/Mixed Analog-Digital Approach
Nomura, O., & Morie, T. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69158-7_111 · Cited by 9
Learning long-range vision for autonomous off-road driving
HadsellRaia, SermanetPierre, BenJan, et al. (2009) · Journal of Field Robotics · doi:10.5555/1507435.1507438 · Cited by 9
Artificial Neural Networks in Biomedicine
Zhou, Z. (2002) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(02)00016-7 · Cited by 7
A multirange architecture for collision-free off-road robot navigation
SermanetPierre, HadsellRaia, ScoffierMarco, et al. (2009) · Journal of Field Robotics · doi:10.5555/1484785.1484787 · Cited by 2
Cited by
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
“Other weight stationary examples are found in [84]–[86], [88], [89].”
BackgroundEyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158
“Among these approaches, FPGA based accelerators have attracted more and more attention of researchers because they have advantages of good performance, high energy efficiency, fast development round, and capability of reconfiguration [1] [2] [3] [6] [12] [14].”
BackgroundMethodInfluentialDianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Going Deeper with Embedded FPGA Platform for Convolutional Neural Network
Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., Song, S., et al. (2016) · Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2847263.2847265 · Cited by 1,252
Metadata
- Authors
- Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S.
- Affiliations
- NEC Laboratories America, Inc., Princeton, NJ, USA
- Venue
- Proceedings of the 37th annual international symposium on Computer architecture
- Publisher
- ACM
- Published
- 2010-06-19
- Pages
- 247-257
- ISSN
- 1063-6897 (ISSN-L)
- OpenAlex
- W2009832130
- Cited by
- 343
- References
- 36
- Citations by year
- 2025: 3 · 2024: 8 · 2023: 11 · 2022: 17 · 2021: 25 · 2020: 46 · 2019: 49 · 2018: 51 · 2017: 56 · 2016: 42 · 2015: 14 · 2014: 9 · 2013: 2 · 2012: 7
- FWCI
- 2.33
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Computer Architecture/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Keywords
- convolutional neural networks, coprocessor architecture, parallelism, off-chip memory bandwidth, object detection, image recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.