CoChat

A Resource Efficient Integer-Arithmetic-Only FPGA-Based CNN Accelerator for Real-Time Facial Emotion Recognition

Open access
Kim, Jaemyung, Kang, Jin-Ku, Kim, Yongwoo · 2021 · IEEE Access

The DOI matches its Crossref record; 53 of the 62 references it cites were found in a registry or an index; 10 of the 34 sources its claims cite were checked for support.

Abstract

Recently, many researches have been conducted on recognition of facial emotion using convolutional neural networks (CNNs), which show excellent performance in computer vision. To obtain a high classification accuracy, a CNN architecture with many parameters and high computational complexity is required. However, this is not suitable for embedded systems where hardware resources are limited. In this paper, we present a lightweight CNN architecture optimized for embedded systems. The proposed CNN architecture has a small memory footprint and low computational complexity. Furthermore, a novel hardware-friendly quantization method that uses only integer-arithmetic is proposed. The proposed hardware-friendly quantization method maps the scale factors to power-of-two terms and replaces multiplication and division operations using scale factors with shift operations. To improve the generalization and classification performance of the CNN, we create the FERPlus-A dataset. This is a new training dataset created using a variety of image processing algorithms. After training with FERPlus-A, quantization has been performed. The size of a quantized CNN parameter is about 0.39 MB, and the number of operations is about 28 M integer operations (IOPs). By evaluating the performance of the quantized CNN that uses only integer-arithmetic on the FERPlus test dataset, the classification accuracy is approximately 86.58%. It achieved higher accuracy than other lightweight CNNs in prior studies. The proposed CNN architecture that uses only integer-arithmetic is implemented on the Xilinx ZC706 SoC platform for real-time facial emotion recognition by applying parallelism strategies and efficient data caching strategies. The FPGA-based CNN accelerator implemented for real-time facial emotion recognition achieves about 10 frame per second (FPS) at 250 MHz and consumes 2.3 W.

Overview

CoChat Summary

TL;DR

The paper proposes a lightweight CNN and log level threshold quantization (LLTQ) for integer-arithmetic-only facial emotion recognition on FPGA. It reports ~86.58% accuracy on the FERPlus test dataset, 0.39 MB parameters, 28 M IOPs, and an FPGA accelerator using 5,090 LUTs, 7,588 FFs, 61 BRAMs, 49 DSPs at ~2.3 W and ~10 FPS.

Key findings

  • FERPlus-A was built by adding 143,545 images to the 28,709 FERPlus training images, resulting in 172,254 images; training on FERPlus-A improved accuracy from ~71.85% to ~86.58% (~14.73% improvement) on the same FERPlus test dataset.
  • With 8-bit quantization-aware training, LSQ improved accuracy by ~0.13% over floating-point but was not hardware-friendly; TQT reduced accuracy by ~0.45%; LLTQ showed no accuracy drop relative to floating-point and used only integer-arithmetic.
  • The proposed CNN has 0.39 MB parameters, 28 M IOPs, and ~86.58% accuracy; compared with lightweight CNNs, it achieved the highest accuracy among the considered methods, e.g., vs [17] it is 6.33× smaller and 3.28% more accurate but uses 1.4× more operations, and vs [23]-3 it is 1.67× smaller, uses 6.07× fewer operations, and is 0.91% more accurate.
  • The FPGA accelerator on Xilinx ZC706 used 5,090 LUTs, 7,588 FFs, 61 BRAMs, and 49 DSPs, consumed ~2.3 W, and achieved ~10 FPS at 250 MHz; INT8 Type-2 vs FP32 Type-1 reduced LUTs by 3.23×, FFs by 4.39×, BRAMs by 1.97×, and DSPs by 4.18×, with total power decreased by 1.87×.

Methods

  • Design flow: FERPlus-A was created by applying bilateral filtering, CLAHE, and sharpening to FERPlus training images; a lightweight CNN was trained in floating-point, then LLTQ quantization-aware training and integer scale conversion were applied; the CNN accelerator IP was designed with HLS and a device driver with C/C++ on a heterogeneous SoC.
  • Architecture: the CNN consists of 21 convolution layers and one global average pooling layer, using modified SqueezeNet fire modules (FireA and FireB), only 3×3 or 1×1 convolutions, power-of-2 feature-map resolution, output channels in multiples of four, ReLU, and BN fusion.
  • Hardware and evaluation: implemented on Xilinx Zynq-7000 SoC ZC706 (dual-core ARM Cortex-A9 PS and XC7Z045 PL) using Vivado HLS 2019.2 at a 250 MHz target; parallelism included intra-convolution fully unrolled MACU (9 multipliers, 8 adders for 3×3) and inter-feature-map parallelism with unroll factor 4, plus FMCM/PCM/OCAM caching and a post-processing module; training used FP batch size 64, 200 epochs, Adam with lr 1e-2 and weight decay 1e-5, and QAT batch size 32, 100 epochs, initial lr 1e-5 for weights/biases and 1e-3 for scale factors/log-thresholds/level-thresholds.

Limitations

  • The paper does not state a limitations section.
  • Evident from the text: compared with [42], the proposed accelerator is more accurate but has a frame rate about 5 FPS slower; compared with [17], it uses 1.4× more operations, though it uses only integer-arithmetic.
  • Evident from the text: evaluation is reported only on the FERPlus test dataset, and hardware results are only for the Xilinx ZC706 platform.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/access.2021.3099075

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

FIGURE 1 An overview of the process for implementing proposed FPGA-based CNN accelerator for real-time facial emotion recognition.
FIGURE 1 An overview of the process for implementing proposed FPGA-based CNN accelerator for real-time facial emotion recognition.CC-BY · From the PDF
FIGURE 2 Proposed lightweight CNN architecture optimized memory usage and computational complexity.
FIGURE 2 Proposed lightweight CNN architecture optimized memory usage and computational complexity.CC-BY · From the PDF
FIGURE 3 The process of a uniform quantizer using the LSQ [38], the TQT [39] method, and the proposed quantization method, LLTQ.
FIGURE 3 The process of a uniform quantizer using the LSQ [38], the TQT [39] method, and the proposed quantization method, LLTQ.CC-BY · From the PDF
Image 4
CC-BY · From the PDF
FIGURE 5 Hardware architecture of the proposed CNN accelerator for real-time facial emotion recognition.
FIGURE 5 Hardware architecture of the proposed CNN accelerator for real-time facial emotion recognition.CC-BY · From the PDF
FIGURE 6 Overall task-scheduling for the proposed CNN accelerator.
FIGURE 6 Overall task-scheduling for the proposed CNN accelerator.CC-BY · From the PDF
Image 7
CC-BY · From the PDF
Image 8
CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Image 10
CC-BY · From the PDF
FIGURE 8 Demonstration of an FPGA-based CNN accelerator using only 8-bit integer-arithmetic for real-time facial emotion recognition.
FIGURE 8 Demonstration of an FPGA-based CNN accelerator using only 8-bit integer-arithmetic for real-time facial emotion recognition.CC-BY · From the PDF
Image 12
CC-BY · From the PDF
Image 13
CC-BY · From the PDF
Image 14
CC-BY · From the PDF

Verification

Claims the paper makes

8 Valid · 2 Not valid · 22 Not checked · 49 Own results and interpretations

62 more claims · 3 with their cited sources checked

Create a free account to see all 81 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 2
  • It may state two separate claims as one. 9
  • The quote only partly states this claim. 4

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 53 references

Research works printed but not linked: 1

  1. Zhu, 2021, THOP: PyTorch-OpCounter

Cited by

See all 50

Metadata

Authors
Kim, J., Kang, J.-K., & Kim, Y.
Affiliations
  • Inha University (KR)
  • Sangmyung University (KR)
Venue
IEEE Access
Published
2021
Volume
9
Pages
104367-104381
ISSN
2169-3536 (ISSN-L)
OpenAlex
W3186146740
Cited by
50
References
62
Citations by year
2026: 5 · 2025: 9 · 2024: 11 · 2023: 14 · 2022: 10 · 2021: 1
FWCI
3.23
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
Keywords
facial emotion recognition, lightweight CNN architecture, embedded systems, real-time inference, shift operations, data caching

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Resource Efficient Integer-Arithmetic-Only FPGA-Based CNN Accelerator for Real-Time Facial Emotion Recognition. https://app.cochat.ai/papers/10.1109/access.2021.3099075
The paper
Kim, J., Kang, J.-K., & Kim, Y. (2021). A Resource Efficient Integer-Arithmetic-Only FPGA-Based CNN Accelerator for Real-Time Facial Emotion Recognition. IEEE Access. https://doi.org/10.1109/access.2021.3099075

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.