CoChat

Layer-Specific Optimization for Mixed Data Flow With Mixed Precision in FPGA Design for CNN-Based Object Detectors

Open access
Nguyen, Duy Thanh, Kim, Hyun, Lee, Hyuk-Jae · 2021 · IEEE Transactions on Circuits and Systems for Video Technology
Cited by 97

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Abstract

Convolutional neural networks (CNNs) require both intensive computation and frequent memory access, which lead to a low processing speed and large power dissipation. Although the characteristics of the different layers in a CNN are frequently quite different, previous hardware designs have employed common optimization schemes for them. This paper proposes a layer-specific design that employs different organizations that are optimized for the different layers. The proposed design employs two layer-specific optimizations: layer-specific mixed data flow and layer-specific mixed precision. The mixed data flow aims to minimize the off-chip access while demanding a minimal on-chip memory (BRAM) resource of an FPGA device. The mixed precision quantization is to achieve both a lossless accuracy and an aggressive model compression, thereby further reducing the off-chip access. A Bayesian optimization approach is used to select the best sparsity for each layer, achieving the best trade-off between the accuracy and compression. This mixing scheme allows the entire network model to be stored in BRAMs of the FPGA to aggressively reduce the off-chip access, and thereby achieves a significant performance enhancement. The model size is reduced by 22.66-28.93 times compared to that in a full-precision network with a negligible degradation of accuracy on VOC, COCO, and ImageNet datasets. Furthermore, the combination of mixed dataflow and mixed precision significantly outperforms the previous works in terms of both throughput, off-chip access, and on-chip memory requirement.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tcsvt.2020.3020569

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 18 references

Research works printed but not linked: 44

  1. gupta, 2015, Deep learning with limited numerical precisions, Proc Int Conf Mach Learn (ICML), 1737
  2. 10.1145/3061639.3062244 · DOI · Paper page
  3. 10.1145/3173162.3173176 · DOI · Paper page
  4. 10.1109/TVLSI.2017.2688340 · DOI · Paper page
  5. 10.1109/HPCA.2017.29 · DOI · Paper page
  6. 10.1145/3079856.3080221 · DOI · Paper page
  7. 10.1109/TCSVT.2018.2864321 · DOI
  8. 10.1109/DAC18072.2020.9218724 · DOI
  9. kang, 2020, Accelerator-aware pruning for convolutional neural networks, IEEE Trans Circuits Syst Video Technol, 30, 2093
  10. 10.23919/DATE.2018.8342033 · DOI
  11. 10.1109/TC.2018.2890258 · DOI
  12. 10.1145/3020078.3021738 · DOI · Paper page
  13. liu, 2015, SSD: Single shot MultiBox detector, arXiv 1512 02325
  14. 10.1109/CVPR.2015.7298594 · DOI · Paper page
  15. redmon, 2018, YOLOv3: An incremental improvement, arXiv 1804 02767
  16. 10.1109/ISCAS.2018.8351021 · DOI · Paper page
  17. 10.1109/TCSI.2019.2962516 · DOI
  18. 10.1145/3079856.3080254 · DOI · Paper page
  19. 10.1145/3289602.3293977 · doi:10.1145/3289602.3293977
  20. zhou, 2017, Incremental network quantization: Towards lossless CNNs with low-precision weights, arXiv 1702 03044
  21. 10.1109/TCSVT.2018.2883087 · DOI
  22. 10.1145/3316781.3317739 · DOI · Paper page
  23. 10.1145/2847263.2847265 · DOI · Paper page
  24. han, 2015, Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding, arXiv 1510 00149 [cs]
  25. 10.1109/CVPR.2018.00474 · DOI · Paper page
  26. 10.1145/3242897 · DOI · Paper page
  27. 10.1145/3240765.3240801 · DOI · Paper page
  28. 10.1109/CVPR.2019.00881 · DOI · Paper page
  29. 10.1016/j.neucom.2017.09.046 · DOI · Paper page
  30. li, 2016, A high performance FPGA-based accelerator for large-scale convolutional neural networks, Proc Int Conf Field Program Logic Appl (FPL), 1
  31. 10.1109/FCCM.2017.64 · DOI · Paper page
  32. 10.1109/TVLSI.2018.2815603 · DOI · Paper page
  33. park, 2018, Value-aware quantization for training and inference of neural networks, arXiv 1804 07802
  34. 10.1109/ISCA.2018.00063 · DOI · Paper page
  35. 10.1109/HPCA.2019.00030 · DOI
  36. 10.1109/ICCV.2017.324 · DOI · Paper page
  37. 10.1145/3174243.3174266 · DOI · Paper page
  38. 10.23919/DATE.2018.8342121 · DOI · Paper page
  39. 10.1145/3289602.3293904 · DOI
  40. 10.1145/3007787.3001163 · DOI · Paper page
  41. 10.1007/s11263-015-0816-y · DOI · Paper page
  42. lin, 0, Microsoft COCO: Common objects in context, Proc Eur Conf Comput Vision (ECCV), 2014, 740
  43. snoek, 2012, Pratical Bayesian optimization of machine algorithms, Proc Adv Neural Inf Process Syst (NIPS), 2951
  44. 2020, BOA The Bayesian optimization algorithm

Cited by

See all 97

Metadata

Authors
Nguyen, D. T., Kim, H., & Lee, H.-J.
Venue
IEEE Transactions on Circuits and Systems for Video Technology
Published
2021-06
Volume
31
Issue
6
Pages
2450-2464
ISSN
  • 1051-8215 (ISSN-L)
  • 1558-2205
OpenAlex
W3082058622
Cited by
97
References
18
FWCI
4.67
Type
journal article
Language
en
Open access
green
Funders
  • Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Ministry of Science and ICT (MSIT), Korea Government (2020-0-01305)
  • Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2019R1A6A1A03032119)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Layer-Specific Optimization for Mixed Data Flow With Mixed Precision in FPGA Design for CNN-Based Object Detectors. https://app.cochat.ai/papers/10.1109/tcsvt.2020.3020569
The paper
Nguyen, D. T., Kim, H., & Lee, H.-J. (2021). Layer-Specific Optimization for Mixed Data Flow With Mixed Precision in FPGA Design for CNN-Based Object Detectors. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/tcsvt.2020.3020569

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.