CoChat
Sign Up
PreprintCited by 58Embedded Systems Design TechniquesAdvanced Neural Network ApplicationsEmbedded Systems and FPGA Applications

ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network

Open access
Gschwend, David·2020·arXiv
DOI 10.48550/arxiv.2005.06892arXiv 2005.06892
Open this paper in CoChat — free accountOpen PDF

Abstract

Image Understanding is becoming a vital feature in ever more applications ranging from medical diagnostics to autonomous vehicles. Many applications demand for embedded solutions that integrate into existing systems with tight real-time and power constraints. Convolutional Neural Networks (CNNs) presently achieve record-breaking accuracies in all image understanding benchmarks, but have a very high computational complexity. Embedded CNNs thus call for small and efficient, yet very powerful computing platforms. This master thesis explores the potential of FPGA-based CNN acceleration and demonstrates a fully functional proof-of-concept CNN implementation on a Zynq System-on-Chip. The ZynqNet Embedded CNN is designed for image classification on ImageNet and consists of ZynqNet CNN, an optimized and customized CNN topology, and the ZynqNet FPGA Accelerator, an FPGA-based architecture for its evaluation. ZynqNet CNN is a highly efficient CNN topology. Detailed analysis and optimization of prior topologies using the custom-designed Netscope CNN Analyzer have enabled a CNN with 84.5% top-5 accuracy at a computational complexity of only 530 million multiplyaccumulate operations. The topology is highly regular and consists exclusively of convolutional layers, ReLU nonlinearities and one global pooling layer. The CNN fits ideally onto the FPGA accelerator. The ZynqNet FPGA Accelerator allows an efficient evaluation of ZynqNet CNN. It accelerates the full network based on a nested-loop algorithm which minimizes the number of arithmetic operations and memory accesses. The FPGA accelerator has been synthesized using High-Level Synthesis for the Xilinx Zynq XC-7Z045, and reaches a clock frequency of 200MHz with a device utilization of 80% to 90 %.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2005.06892

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 53

  1. [1]F. Conti and L. Benini, 'A ultra-low-energy convolution engine for fast brain-inspired vision in multicore clusters', in 2015 Design, Automation Test in Europe Conference Exhibition (DATE) , 2015, pp. 683-688 (cit. on p. 1).
  2. [2]W. S. McCulloch and W. Pitts, 'A logical calculus of the ideas immanent in nervous activity', The bulletin of mathematical biophysics , vol. 5, no. 4, pp. 115-133, 1943 (cit. on p. 1).
  3. [6]N. Gupta, 'Machine learning in the cloud: Deep neural networks on FPGAs', Xcell Journal , vol. 92, pp. 46-51, Sep. 2015 (cit. on p. 2).
  4. [12]I. Goodfellow, Y. Bengio, and A. Courville, 'Deep learning', Book in preparation for MIT Press, 2016 (cit. on pp. 3, 5).
  5. [15]S. Herculano-Houzel, 'The human brain in numbers: A linearly scaled-up primate brain', Frontiers in Human Neuroscience , vol. 3, no. 31, 2009 (cit. on p. 3).
  6. [17]D. Gschwend, C. Mayer, and S. Willi, 'Origami: Design and implementation of a convolutional neural network accelerator ASIC', Semester Thesis, ETH Zürich, 2015 (cit. on pp. 4, 6, 17).
  7. [18]S. Hochreiter and J. Schmidhuber, 'Long short-term memory', Neural computation , vol. 9, no. 8, pp. 1735-1780, 1997 (cit. on p. 4).
  8. [19]H. Sak, A. W. Senior, and F. Beaufays, 'Long short-term memory recurrent neural network architectures for large scale acoustic modeling.', in INTERSPEECH , 2014, pp. 338-342 (cit. on p. 4).
  9. [20]Y. LeCun, Y. Bengio, and G. Hinton, 'Deep learning', Nature , vol. 521, no. 7553, pp. 436-444, 2015 (cit. on p. 5).
  10. [21]C. Farabet, B. Martini, P. Akselrod, et al. , 'Hardware accelerated convolutional neural networks for synthetic vision systems', in Proceedings of 2010 IEEE International Symposium on Circuits and Systems , IEEE, 2010, pp. 257-260 (cit. on p. 6).
  11. [22]K. He, X. Zhang, S. Ren, and J. Sun, 'Delving deep into rectifiers: Surpassing human-level performance on imagenet classification', in Proceedings of the IEEE International Conference on Computer Vision , 2015, pp. 1026-1034 (cit. on p. 7).
  12. [23]I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, 'Maxout Networks', ArXiv e-prints , Feb. 2013. arXiv: 1302.4389 [stat.ML] (cit. on p. 7). · arXiv
  13. [24]D. Clevert, T. Unterthiner, and S. Hochreiter, 'Fast and accurate deep network learning by exponential linear units (elus)', CoRR , vol. abs/1511.07289, 2015 (cit. on p. 7). · arXiv
  14. [25]A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'ImageNet classification with deep convolutional neural networks', in Advances in Neural Information Processing Systems 25 , F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds., Curran Associates, Inc., 2012, pp. 1097-1105 (cit. on pp. 7, 9).
  15. [26]S. Ioffe and C. Szegedy, 'Batch normalization: Accelerating deep network training by reducing internal covariate shift', CoRR , vol. abs/1502.03167, 2015 (cit. on p. 7). · arXiv
  16. [28]R. Al-Rfou, G. Alain, A. Almahairi, et al. , 'Theano: A python framework for fast computation of mathematical expressions', CoRR , vol. abs/1605.02688, 2016 (cit. on p. 8). · arXiv
  17. [31]R. Collobert, K. Kavukcuoglu, and C. Farabet, 'Torch7: A Matlab-like environment for machine learning', in BigLearn, NIPS Workshop , 2011 (cit. on p. 8).
  18. [33]Y. Jia, E. Shelhamer, J. Donahue, et al. , 'Caffe: Convolutional architecture for fast feature embedding', ArXiv preprint arXiv:1408.5093 , 2014 (cit. on p. 8). · arXiv
  19. [35]O. Russakovsky, J. Deng, H. Su, et al. , 'ImageNet Large Scale Visual Recognition Challenge', International Journal of Computer Vision (IJCV) , vol. 115, no. 3, pp. 211-252, 2015 (cit. on p. 9).
  20. [36]F. N. Iandola, K. Ashraf, M. W. Moskewicz, and K. Keutzer, 'Firecaffe: Near-linear acceleration of deep neural network training on compute clusters', CoRR , vol. abs/1511.00175, 2015 (cit. on p. 10). · arXiv
  21. [37]M. Lin, Q. Chen, and S. Yan, 'Network in network', CoRR , vol. abs/1312.4400, 2013 (cit. on p. 10). · arXiv
  22. [38]K. Simonyan and A. Zisserman, 'Very deep convolutional networks for large-scale image recognition', ArXiv preprint arXiv:1409.1556 , 2014 (cit. on p. 10). · arXiv
  23. [40]C. Szegedy, W. Liu, Y. Jia, et al. , 'Going deeper with convolutions', in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2015, pp. 1-9 (cit. on p. 10).
  24. [41]K. He, X. Zhang, S. Ren, and J. Sun, 'Deep residual learning for image recognition', ArXiv preprint arXiv:1512.03385 , 2015 (cit. on p. 11). · arXiv
  25. [42]C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, 'Rethinking the inception architecture for computer vision', CoRR , vol. abs/1512.00567, 2015 (cit. on p. 11). · arXiv
  26. [43]C. Szegedy, S. Ioffe, and V. Vanhoucke, 'Inception-v4, Inception-ResNet and the impact of residual connections on learning', CoRR , vol. abs/1602.07261, 2016 (cit. on p. 11). · arXiv
  27. [44]F. N. Iandola, M. W. Moskewicz, K. Ashraf, et al. , 'SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1mb model size', ArXiv:1602.07360 , 2016 (cit. on pp. 11, 26). · arXiv
  28. [45]M. Denil, B. Shakibi, L. Dinh, M. Ranzato, and N. de Freitas, 'Predicting parameters in deep learning', CoRR , vol. abs/1306.0543, 2013 (cit. on p. 11). · arXiv
  29. [46]E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, 'Exploiting linear structure within convolutional networks for efficient evaluation', in Advances in Neural Information Processing Systems , 2014, pp. 1269-1277 (cit. on p. 11).
  30. [47]J. Jin, A. Dundar, and E. Culurciello, 'Flattened convolutional neural networks for feedforward acceleration', CoRR , vol. abs/1412.5474, 2014 (cit. on p. 11). · arXiv
  31. [48]Y. Kim, E. Park, S. Yoo, et al. , 'Compression of deep convolutional neural networks for fast and low power mobile applications', CoRR , vol. abs/1511.06530, 2015 (cit. on p. 11). · arXiv
  32. [50]S. Anwar, K. Hwang, and W. Sung, 'Structured pruning of deep convolutional neural networks', ArXiv preprint arXiv:1512.08571 , 2015 (cit. on p. 11). · arXiv
  33. [54]S. Anwar, K. Hwang, and W. Sung, 'Fixed point optimization of deep convolutional neural networks for object recognition', in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , IEEE, 2015, pp. 1131-1135 (cit. on p. 11).
  34. [55]W. Sung, S. Shin, and K. Hwang, 'Resiliency of deep neural networks under quantization', ArXiv preprint arXiv:1511.06488 , 2015 (cit. on p. 12). · arXiv
  35. [56]M. Courbariaux and Y. Bengio, 'BinaryNet: Training deep neural networks with weights and activations constrained to +1 or -1', CoRR , vol. abs/1602.02830, 2016 (cit. on pp. 12, 32). · arXiv
  36. [57]M. Courbariaux, Y. Bengio, and J. David, 'Binaryconnect: Training deep neural networks with binary weights during propagations', CoRR , vol. abs/1511.00363, 2015 (cit. on pp. 12, 32). · arXiv
  37. [58]P. Gysel, M. Motamedi, and S. Ghiasi, 'Hardware-oriented approximation of convolutional neural networks', ArXiv preprint arXiv:1604.03168 , 2016 (cit. on pp. 12, 33). · arXiv
  38. [59]S. Han, H. Mao, and W. J. Dally, 'A deep neural network compression pipeline: Pruning, quantization, huffman encoding', ArXiv preprint arXiv:1510.00149 , 2015 (cit. on p. 12). · arXiv
  39. [60]H. Kaeslin, Digital Integrated Circuit Design: From VLSI Architectures to CMOS Fabrication , 1st ed. Cambridge University Press, 2008 (cit. on pp. 13, 14).
  40. [67]M. Fingeroff, High-Level Synthesis Blue Book . Xlibris Corporation, 2010 (cit. on pp. 14, 40, 44).
  41. [79]S. Liu, Analog VLSI: Circuits and principles , ser. Bradford Book. MIT Press, 2002 (cit. on p. 16).
  42. [81]G. Lacey, G. W. Taylor, and S. Areibi, 'Deep learning on FPGAs: Past, present, and future', ArXiv preprint arXiv:1602.04283 , 2016 (cit. on p. 16). · arXiv
  43. [83]K. Ovtcharov, O. Ruwase, J.-Y. Kim, et al. , Accelerating deep convolutional neural networks using specialized hardware , Feb. 2015 (cit. on pp. 17, 18).
  44. [86]Y. Chen, T. Luo, S. Liu, et al. , 'DaDianNao: A machine-learning supercomputer', in Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture , IEEE Computer Society, 2014, pp. 609-622 (cit. on p. 17).
  45. [87]L. Cavigelli, D. Gschwend, C. Mayer, et al. , 'Origami: A convolutional network accelerator', in Proceedings of the 25th Edition on Great Lakes Symposium on VLSI , ser. GLSVLSI '15, Pittsburgh, Pennsylvania, USA: ACM, 2015, pp. 199-204 (cit. on p. 17).
  46. [89]Y.-H. Chen, T. Krishna, J. Emer, and V. Sze, 'Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks', in IEEE International Solid-State Circuits Conference, ISSCC 2016, Digest of Technical Papers , IEEE Computer Society, 2016, pp. 262-263 (cit. on p. 18).
  47. [90]C. Zhang, P. Li, G. Sun, et al. , 'Optimizing FPGA-based accelerator design for deep convolutional neural networks', in Proceedings of the 2015 ACM/SIGDA International Symposium on FieldProgrammable Gate Arrays , ACM, 2015, pp. 161-170 (cit. on p. 18).
  48. [91]M. Motamedi, P. Gysel, V. Akella, and S. Ghiasi, 'Design space exploration of FPGA-based deep convolutional neural networks', in 2016 21st Asia and South Pacific Design Automation Conference (ASP-DAC) , IEEE, 2016, pp. 575-580 (cit. on p. 18).
  49. [96]J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, 'Striving for simplicity: The all convolutional net', ArXiv preprint arXiv:1412.6806 , 2014 (cit. on pp. 23, 28). · arXiv
  50. [104]D. Mishkin, N. Sergievskiy, and J. Matas, 'Systematic evaluation of CNN advances on the ImageNet', ArXiv e-prints , Jun. 2016. arXiv: 1606.02228 (cit. on pp. 29, 82). · arXiv
  51. [110]C. R. Wan and D. J. Evans, 'Nineteen ways of systolic matrix multiplication', International Journal of Computer Mathematics , vol. 68, no. 1-2, pp. 39-69, 1998 (cit. on p. 36).
  52. [111]N. Vasilache, J. Johnson, M. Mathieu, et al. , 'Fast convolutional nets with fbfft: A GPU performance evaluation', CoRR , vol. abs/1412.7580, 2014 (cit. on p. 36). · arXiv
  53. [129]A. Krizhevsky, 'One weird trick for parallelizing convolutional neural networks', CoRR , vol. abs/1404.5997, 2014 (cit. on p. 82). · arXiv

Cited by

Sorted by citations

Authors
Gschwend, D.
Corresponding author
Gschwend, David
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2005.06892
arXiv
2005.06892
OpenAlex
W3026022605
Cited by
58
References
0
Citations by year
2025: 3 · 2024: 2 · 2023: 5 · 2022: 8 · 2021: 19 · 2020: 8 · 2019: 9 · 2018: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
Keywords
convolutional neural networks, FPGA acceleration, embedded machine learning, Zynq system-on-chip, image classification, ImageNet, high-level synthesis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review