The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Facial Emotion Recognition Using Transfer Learning in the Deep CNN
Open accessAbstract
Human facial emotion recognition (FER) has attracted the attention of the research community for its promising applications. Mapping different facial expressions to the respective emotional states are the main task in FER. The classical FER consists of two major steps: feature extraction and emotion recognition. Currently, the Deep Neural Networks, especially the Convolutional Neural Network (CNN), is widely used in FER by virtue of its inherent feature extraction mechanism from images. Several works have been reported on CNN with only a few layers to resolve FER problems. However, standard shallow CNNs with straightforward learning schemes have limited feature extraction capability to capture emotion information from high-resolution images. A notable drawback of the most existing methods is that they consider only the frontal images (i.e., ignore profile views for convenience), although the profile views taken from different angles are important for a practical FER system. For developing a highly accurate FER system, this study proposes a very Deep CNN (DCNN) modeling through Transfer Learning (TL) technique where a pre-trained DCNN model is adopted by replacing its dense upper layer(s) compatible with FER, and the model is fine-tuned with facial emotion data. A novel pipeline strategy is introduced, where the training of the dense layer(s) is followed by tuning each of the pre-trained DCNN blocks successively that has led to gradual improvement of the accuracy of FER to a higher level. The proposed FER system is verified on eight different pre-trained DCNN models (VGG-16, VGG-19, ResNet-18, ResNet-34, ResNet-50, ResNet-152, Inception-v3 and DenseNet-161) and well-known KDEF and JAFFE facial image datasets. FER is very challenging even for frontal views alone. FER on the KDEF dataset poses further challenges due to the diversity of images with different profile views together with frontal views. The proposed method achieved remarkable accuracy on both datasets with pre-trained models. On a 10-fold cross-validation way, the best achieved FER accuracies with DenseNet-161 on test sets of KDEF and JAFFE are 96.51% and 99.52%, respectively. The evaluation results reveal the superiority of the proposed FER system over the existing ones regarding emotion detection accuracy. Moreover, the achieved performance on the KDEF dataset with profile views is promising as it clearly demonstrates the required proficiency for real-life applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
Baio, J., Wiggins, L., Christensen, D. L., Maenner, M. J., Daniels, J., Warren, Z., Kurzius-Spencer, M., Zahorodny, W., Robinson, C., Rosenberg, et al. (2018) · MMWR. Surveillance Summaries · doi:10.15585/mmwr.ss6706a1 · Cited by 6,078
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509
Maenner, M. J., Shaw, K. A., Baio, J., EdS1, Washington, A., Patrick, M., DiRienzo, M., Christensen, D. L., Wiggins, L. D., Pettygrove, S., et al. (2020) · MMWR. Surveillance Summaries · doi:10.15585/mmwr.ss6904a1 · Cited by 3,780
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503 · cited 3×
“From a practical point of view, both selecting a pre-trained model and determining a size-similarity matrix for fine-tuning are important in TL [40,70].”
BackgroundLearning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150 · cited 2×
“However, research has shown that TL [37,38] can be very useful to solve this issue.”
BackgroundMethod
Cited by
Deep Residual Learning for Image Recognition: A Survey
Shafiq, M., & Gu, Z. (2022) · Applied Sciences · doi:10.3390/app12188972 · Cited by 1,023
A Review of Deep Transfer Learning and Recent Advancements
Iman, M., Arabnia, H. R., & Rasheed, K. (2023) · Technologies · doi:10.3390/technologies11020040 · Cited by 680
Convolutional Neural Networks: A Survey
Krichen, M. (2023) · Computers · doi:10.3390/computers12080151 · Cited by 623
Khare, S. K., Blanes-Vidal, V., Nadimi, E. S., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102019 · Cited by 469
Fine-grained activity classification in assembly based on multi-visual modalities
Chen, H., Zendehdel, N., Leu, M. C., & Yin, Z. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02152-x · Cited by 47
Metadata
- Authors
- Akhand, M. A. H., Roy, S., Siddique, N., Kamal, M. A. S., & Shimamura, T.
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2021-04-27
- Volume
- 10
- Issue
- 9
- Pages
- 1036
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W3157429286
- Cited by
- 333
- References
- 11
- FWCI
- 51.54
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.