The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Efficient Facial Expression Recognition Algorithm Based on Hierarchical Deep Neural Network Structure
Open accessAbstract
With the continued development of artificial intelligence (AI) technology, research on interaction technology has become more popular. Facial expression recognition (FER) is an important type of visual information that can be used to understand a human's emotional situation. In particular, the importance of AI systems has recently increased due to advancements in research on AI systems applied to AI robots. In this paper, we propose a new scheme for FER system based on hierarchical deep learning. The feature extracted from the appearance feature-based network is fused with the geometric feature in a hierarchical structure. The appearance feature-based network extracts holistic features of the face using the preprocessed LBP image, whereas the geometric feature-based network learns the coordinate change of action units (AUs) landmark, which is a muscle that moves mainly when making facial expressions. The proposed method combines the result of the softmax function of two features by considering the error associated with the second highest emotion (Top-2) prediction result. In addition, we propose a technique to generate facial images with neutral emotion using the autoencoder technique. By this technique, we can extract the dynamic facial features between the neutral and emotional images without sequence data. We compare the proposed algorithm with the other recent algorithms for CK+ and JAFFE dataset, which are typically considered to be verified datasets in the facial expression recognition. The ten-fold cross validation results show 96.46% of accuracy in the CK+ dataset and 91.27% of accuracy in the JAFFE dataset. When comparing with other methods, the result of the proposed hierarchical deep network structure shows up to about 3% of the accuracy improvement and 1.3% of average improvement in CK+ dataset, respectively. In JAFFE datasets, up to about 7% of the accuracy is enhanced, and the average improvement is verified by about 1.5%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2014) · Cited by 33,419
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Bilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“We used a preprocessing technique involving bilateral blurring [25].”
MethodCootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831
Dlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Coding facial expressions with Gabor wavelets
Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670949 · Cited by 2,210
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810
Fast Human Detection Using a Cascade of Histograms of Oriented Gradients
Zhu, Q., Yeh, M.-C., Cheng, K.-T., & Avidan, S. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.119 · Cited by 1,399
Local Binary Patterns and Its Application to Facial Image Analysis: A Survey
Huang, D., Shan, C., Ardabilian, M., Wang, Y., & Chen, L. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2118750 · Cited by 905
Lopes, A. T., de Aguiar, E., De Souza, A. F., & Oliveira-Santos, T. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.07.026 · Cited by 799
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.341 · Cited by 781
Self-citationKotsia, I., & Pitas, I. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.884954 · Cited by 687
300 Faces In-The-Wild Challenge: Database and results
Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2016) · Image and Vision Computing · doi:10.1016/j.imavis.2016.01.002 · Cited by 638
Automatic facial expression recognition using features of salient facial patches
Happy, S. L., & Routray, A. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2386334 · Cited by 599
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks
Zhang, K., Huang, Y., Du, Y., & Wang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2689999 · Cited by 487
Automatic Analysis of Facial Actions: A Survey
Martinez, B., Valstar, M. F., Jiang, B., & Pantic, M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2731763 · Cited by 350
Facial feature detection using Haar classifiers
Wilson, P., & Fernandez, J. D. (2006) · doi:10.5555/1127389.1127416 · Cited by 340
Facial expression recognition using radial encoding of local Gabor features and classifier synthesis
Gu, W., Xiang, C., Venkatesh, Y. V., Huang, D., & Lin, H. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.05.006 · Cited by 311
A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models
Koelstra, S., Pantic, M., & Patras, I. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.50 · Cited by 242
Xie, S., & Hu, H. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2844085 · Cited by 214
Yang, B., Cao, J., Ni, R., & Zhang, Y. (2018) · IEEE Access · doi:10.1109/access.2017.2784096 · Cited by 204
Local Directional Ternary Pattern for Facial Expression Recognition
Ryu, B., Rivera, A. R., Kim, J., & Chae, O. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2726010 · Cited by 150
Self-citationFacial Expression Recognition with Fusion Features Extracted from Salient Facial Areas
Liu, Y., Li, Y., Ma, X., & Song, R. (2017) · Sensors · doi:10.3390/s17040712 · Cited by 88
Facial animation parameters extraction and expression recognition using Hidden Markov Models
Pardàs, M., & Bonafonte, A. (2002) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(02)00078-4 · Cited by 81
A spatio-temporal RBM-based model for facial expression recognition
Elaiwat, S., Bennamoun, M., & Boussaïd, F. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.07.006 · Cited by 71
Facial expression recognition with PCA and LBP features extracting from active facial patches
Liu, Y., Cao, Y., Li, Y., Liu, M., Song, R., Wang, Y., Xu, Z., & Ma, X. (2016) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar.2016.7784056 · Cited by 35
Multi-Level Haar Wavelet based Facial Expression Recognition using Logistic Regression
Goyani, M. M., & Patel, N. M. (2017) · Indian Journal of Science and Technology · doi:10.17485/ijst/2017/v10i9/108944 · Cited by 23
Cited by
Khare, S. K., Blanes-Vidal, V., Nadimi, E. S., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102019 · Cited by 469
Recognition of facial expression of fetuses by artificial intelligence (AI)
Miyagi, Y., Hata, T., Bouno, S., Koyanagi, A., & Miyake, T. (2021) · Journal of Perinatal Medicine · doi:10.1515/jpm-2020-0537 · Cited by 34
Miyagi, Y., Hata, T., Bouno, S., Koyanagi, A., & Miyake, T. (2023) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.14569 · Cited by 15
Metadata
- Authors
- Kim, J.-H., Kim, B.-G., Roy, P. P., & Jeong, D.-M.
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 41273-41285
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2931011950
- Cited by
- 303
- References
- 34
- FWCI
- 48.68
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.