Multi-Objective Based Spatio-Temporal Feature Representation Learning Robust to Expression Intensity Variations for Facial Expression Recognition
Abstract
Facial expression recognition (FER) is increasingly gaining importance in various emerging affective computing applications. In practice, achieving accurate FER is challenging due to the large amount of inter-personal variations such as expression intensity variations. In this paper, we propose a new spatio-temporal feature representation learning for FER that is robust to expression intensity variations. The proposed method utilizes representative expression-states (e.g., onset, apex and offset of expressions) which can be specified in facial sequences regardless of the expression intensity. The characteristics of facial expressions are encoded in two parts in this paper. As the first part, spatial image characteristics of the representative expression-state frames are learned via a convolutional neural network. Five objective terms are proposed to improve the expression class separability of the spatial feature representation. In the second part, temporal characteristics of the spatial feature representation in the first part are learned with a long short-term memory of the facial expression. Comprehensive experiments have been conducted on a deliberate expression dataset (MMI) and a spontaneous micro-expression dataset (CASME II). Experimental results showed that the proposed method achieved higher recognition rates in both datasets compared to the state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537 · see citing works
(2010) · Cited by 14,134 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287 · see citing works
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · see citing works
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikäinen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,592 · see citing works
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846 · see citing works
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299101 · Cited by 2,121 · see citing works
Ekman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,975 · see citing works
Face recognition by independent component analysis
Bartlett, M. S., Movellan, J. R., & B, T. J. S. A. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.804287 · Cited by 1,931 · see citing works
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810 · see citing works
Going deeper in facial expression recognition using deep neural networks
Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477450 · Cited by 1,132 · see citing works
Fast Normalized Cross-Correlation
Yoo, J.-C., & Han, T. H. (2009) · Circuits Systems and Signal Processing · doi:10.1007/s00034-009-9130-7 · Cited by 1,128 · see citing works
Web-Based Database for Facial Expression Analysis
Pantić, M., Valstar, M., Rademaker, R. M., & Maat, L. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521424 · Cited by 1,099 · see citing works
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999 · see citing works
CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation
Yan, W.-J., Li, X., Wang, S.-J., Zhao, G., Liu, Y.-J., Chen, Y.-H., & Fu, X. (2014) · PLoS ONE · doi:10.1371/journal.pone.0086041 · Cited by 993 · see citing works
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.341 · Cited by 781 · cited 6× · see citing works
“In [32], a temporal geometric feature was jointly learned in order to reduce the effect of the identity on the learned spatio-temporal appearance features.”
BackgroundMethodInfluentialKotsia, I., & Pitas, I. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.884954 · Cited by 687 · see citing works
PCCA: A new approach for distance learning from sparse pairwise constraints
Mignon, A., & Jurie, F. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247987 · Cited by 621 · see citing works
Action Recognition using Visual AttentionPreprint
Sharma, S., Kiros, R., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04119 · Cited by 593 · see citing works
Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network
Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., & Nielsen, M. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_31 · Cited by 582 · see citing works
The computer expression recognition toolbox (CERT)
Littlewort, G. C., Whitehill, J., Wu, T., Fasel, I., Frank, M. G., Movellan, J. R., & Bartlett, M. S. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771414 · Cited by 550 · see citing works
Local Directional Number Pattern for Face Analysis: Face and Expression Recognition
Rivera, A. R., Castillo, J. R., & Chae, O. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2235848 · Cited by 512 · see citing works
Incremental Face Alignment in the Wild
Asthana, A., Zafeiriou, S. P., Cheng, S., & Pantić, M. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.240 · Cited by 453 · see citing works
Notes on Convolutional Neural NetworksPreprint
Bouvrie, J. (2006) · CogPrints (University of Southampton) · Cited by 436 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407 · see citing works
Recurrent Neural Networks for Emotion Recognition in Video
Kahou, S. E., Michalski, V., Konda, K., Memisevic, R., & Pal, C. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2830596 · Cited by 386 · see citing works
A real-time automated system for the recognition of human facial expressions
Anderson, K., & McOwan, P. W. (2006) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2005.854502 · Cited by 351 · see citing works
Fully Automatic Facial Action Unit Detection and Temporal Analysis
Valstar, M., & Pantić, M. (2006) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2006.85 · Cited by 349 · see citing works
Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis
Liu, M., Li, S., Shan, S., Wang, R., & Chen, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16817-3_10 · Cited by 332 · see citing works
Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?
Khorrami, P., Paine, T. L., & Huang, T. S. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.12 · Cited by 285 · see citing works
Fast normalized cross correlation for defect detection
Tsai, D., & Lin, C.-T. (2003) · Pattern Recognition Letters · doi:10.1016/s0167-8655(03)00106-5 · Cited by 281 · see citing works
Gabor wavelets and General Discriminant Analysis for face identification and verification
Shen, L., Bai, L., & Fairhurst, M. C. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2006.05.002 · Cited by 266 · see citing works
Action unit detection using sparse appearance descriptors in space-time video volumes
Jiang, B., Valstar, M., & Pantić, M. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771416 · Cited by 236 · see citing works
AU-aware Deep Networks for facial expression recognition
Liu, M., Li, S., Shan, S., & Chen, X. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553734 · Cited by 229 · see citing works
AU-inspired Deep Networks for Facial Expression Feature Learning
Liu, M., Li, S., Shan, S., & Chen, X. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.011 · Cited by 222 · see citing works
Cross-entropy vs. squared error training: a theoretical and experimental comparison
Golik, P., Doetsch, P., & Ney, H. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-436 · Cited by 221 · see citing works
Evaluation of Face Resolution for Expression Analysis
Tian, Y. (2005) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvpr.2004.334 · Cited by 207 · see citing works
A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling
Jiang, B., Valstar, M., Martínez, B., & Pantić, M. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2249063 · Cited by 146 · see citing works
Efficient Spatio-Temporal Local Binary Patterns for Spontaneous Facial Micro-Expression Recognition
Wang, Y., See, J., Phan, R. C. ‐., & Oh, Y.-H. (2015) · PLoS ONE · doi:10.1371/journal.pone.0124674 · Cited by 143 · see citing works
Long Short Term Memory Recurrent Neural Network based Multimodal Dimensional Emotion Recognition
Chao, L., Tao, J., Yang, M., Li, Y., & Wen, Z. (2015) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2808196.2811634 · Cited by 124 · see citing works
Lee, S. H., Plataniotis, K. N., & Ro, Y. M. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2346515 · Cited by 124 · see citing works
Self-citationHow deep neural networks can improve emotion recognition on video data
Khorrami, P., Le Paine, T., Brady, K., Dagli, C., & Huang, T. S. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7532431 · Cited by 107 · see citing works
A new method for facial expression recognition based on sparse representation plus LBP
Huang, M., Wang, Z., & Ying, Z. (2010) · International Congress on Image and Signal Processing (CISP) · doi:10.1109/cisp.2010.5647898 · Cited by 74 · see citing works
Robust Facial Expression Recognition via Compressive Sensing
Zhang, S., Zhao, X., & Lei, B. (2012) · Sensors · doi:10.3390/s120303747 · Cited by 73 · see citing works
Sparse representations for facial expressions recognition via l1 optimization
Zafeiriou, S. P., & Petrou, M. (2010) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2010.5543148 · Cited by 71 · see citing works
Facial Expression Recognition Based on Local Phase Quantization and Sparse Representation
Wang, Z., & Ying, Z. (2012) · International Conference on Natural Computation (ICNC) · doi:10.1109/icnc.2012.6234551 · Cited by 70 · see citing works
Quantifying Micro-expressions with Constraint Local Model and Local Binary Pattern
Yan, W., Wang, S., Chen, Y.-H., Zhao, G., & Fu, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16178-5_20 · Cited by 58 · see citing works
Monogenic Riesz wavelet representation for micro-expression recognition
Oh, Y.-H., Ngo, A. C. L., See, J., Liong, S., Phan, R. C. ‐., & Ling, H.-C. (2015) · IEEE International Conference on Digital Signal Processing (DSP) · doi:10.1109/icdsp.2015.7252078 · Cited by 58 · see citing works
Chan, C. H., Kittler, J., Poh, N., Ahonen, T., & Pietikäinen, M. (2009) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2009.5457642 · Cited by 58 · see citing works
Lee, S. H., Baddar, W. J., & Ro, Y. M. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.12.016 · Cited by 54 · see citing works
Self-citationSubtle Facial Expression Recognition Using Adaptive Magnification of Discriminative Facial Motion
Park, S., Lee, S. H., & Ro, Y. M. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806362 · Cited by 41 · see citing works
Self-citationFacial expression recognition using geometric and appearance features
Chen, J., Chen, D., Gong, Y., Meng, Y., Zhang, K., & Wang, L. (2012) · International Conference on Internet Multimedia Computing and Service · doi:10.1145/2382336.2382345 · Cited by 39 · see citing works
Lee, S. H., & Ro, Y. M. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2496320 · Cited by 31 · see citing works
Self-citationAutomatic Facial Expression Recognition Using Extended AR-LBP
Naika, C. L. S., Jha, S. S., Das, P. K., & Nair, S. B. (2012) · Communications in computer and information science · doi:10.1007/978-3-642-31686-9_29 · Cited by 8 · see citing works
Flagged first, then by citations
3 of the 61 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,043 · see citing works
“In [71], more intensity states (onset, onset to apex transition, apex, apex to offset transition and offset)”
BackgroundA Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. C. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 747 · see citing works
“Therefore, this sub-section briefly introduces some popular databases related to FER consisting of 2D and 3D video sequences and still images:”
BackgroundA systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 657 · see citing works
Facial emotion recognition using deep learning: review and insights
Mellouk, W., & Handouzi, W. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.07.101 · Cited by 354 · see citing works
Zhang, S., Yang, Y., Chen, C., Zhang, X., Leng, Q., & Zhao, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121692 · Cited by 273 · see citing works
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
Leveraging recent advances in deep learning for audio-Visual emotion recognition
Schoneveld, L., Othmani, A., & Abdelkawy, H. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.03.007 · Cited by 243 · see citing works
“It has been demonstrated that deep neural networks and especially recurrent neural networks are capable of capturing the temporal dynamics of such sequences (Kim et al. [2017]).”
BackgroundZhang, X., Liu, J., Shen, J. G., Li, S., Hou, K., Hu, B., Gao, J. G., Zhang, T., & Hu, B. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2987575 · Cited by 209 · see citing works
Karnati, M., Seal, A., Krejcar, O., & Yazidi, A. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3031835 · Cited by 196 · see citing works
“D Inception-ResNet + landmarks [56], autoen-coders + SOM [57], spatiotemporal feature + LSTM [58], ensemble DCNNs [59], DCNN for binary classification [60], IACNN [61], 2B(N + M)Softmax [62], S-DSRN [63], color features + Gabor transform [64], DCMA-CNNs [65], broad learning [66], ensemble MLCNNs…”
BackgroundMethodFacial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195 · see citing works
“[96] take advantage of CNN to learn spatial features of representative state frames, and then the LSTM of a facial expression is employed to learn the temporal features of the spatial feature representation.”
MethodFER-net: facial expression recognition using deep neural net
Karnati, M., Seal, A., Krejcar, O., & Yazidi, A. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05676-y · Cited by 159 · see citing works
Pan, B., Hirota, K., Jia, Z., & Dai, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126866 · Cited by 142 · see citing works
Cai, Y., Li, X., & Li, J. (2023) · Sensors · doi:10.3390/s23052455 · Cited by 143 · see citing works
“Different individuals have different skin colors, looks, and facial features [36,37], which pose challenges to the accuracy of classification.”
BackgroundReddy, S., Karri, S. T., Dubey, S. R., & Mukherjee, S. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852419 · Cited by 119 · see citing works
“a two-step deep learning architecture [32], [35], [36], [37].”
BackgroundExpression snippet transformer for robust video-based facial expression recognition
Liu, Y., Wang, W., Feng, C., Zhang, H., Chen, Z., & Zhan, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109368 · Cited by 124 · see citing works
Recognizing Facial Expressions Using a Shallow Convolutional Neural Network
Miao, S., Xu, H., Han, Z., & Zhu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2921220 · Cited by 110 · see citing works
Transformer-based Multimodal Information Fusion for Facial Expression Analysis
Zhang, W., Qiu, F., Wang, S., Zeng, H., Zhang, Z., An, R., Ma, B., & Ding, Y. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00271 · Cited by 109 · see citing works
“Some other works [6, 17] utilize LSTM to model the temporal dependencies and characteristics within consecutive facial images.”
STAN: Spatio-Temporal Adversarial Networks for Abnormal Event Detection
Lee, S., Kim, H. G., & Ro, Y. M. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462388 · Cited by 104 · see citing works
Self-citationCogEmoNet: A Cognitive-Feature-Augmented Driver Emotion Recognition Model for Smart Cockpit
Li, W., Zeng, G., Zhang, J., Xu, Y. H., Xing, Y., Zhou, R., Guo, G., Shen, Y., Cao, D., & Wang, F. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3127935 · Cited by 104 · see citing works
“Cross entropy (CE) [39], F1 [40], mean square error (MSE) [41], and consistency correlation coefficient (CCC) [34] loss function are applied to optimize accuracy (Acc), F1 score, MSE, and CCC respectively.”
InfluentialDeep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition
Jeong, D.-M., Kim, B., & Dong, S.-Y. (2020) · Sensors · doi:10.3390/s20071936 · Cited by 102 · see citing works
“It is even less than STFR model [83] and 3DIR model [11].”
MethodOAENet: Oriented attention ensemble for accurate facial expression recognition
Wang, Z., Zeng, F., Liu, S., & Zeng, B. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107694 · Cited by 104 · see citing works
A comprehensive review of facial expression recognition techniques
Rashmi Adyapady, R., & Annappa, B. (2022) · Multimedia Systems · doi:10.1007/s00530-022-00984-w · Cited by 98 · see citing works
“The FER systems can be divided into two categories, the work considering the static images and those that work with dynamic image sequences [40].”
BackgroundInfluentialMicro-expression recognition: an updated review of current trends, challenges and solutions
Goh, K. M., Ng, C. H., Lim, L. L., & Sheikh, U. U. (2018) · The Visual Computer · doi:10.1007/s00371-018-1607-6 · Cited by 92 · see citing works
“Within high-level approach classification in ME recognition; fully connected layer [80, 81] in CNN is used.”
BackgroundMethodInfluentialMuzammel, M., Salam, H., & Othmani, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106433 · Cited by 88 · see citing works
Clip-aware expressive feature learning for video-based facial expression recognition
Liu, Y., Feng, C., Yuan, X., Zhou, L., Wang, W., Qin, J., & Luo, Z. (2022) · Information Sciences · doi:10.1016/j.ins.2022.03.062 · Cited by 83 · see citing works
“…73.23 WMDCNN [42] frame-based 78.2 CER [7] peak frame-based 70.12 AUDN [41] peak frame-based 75.85 LPQ-TOP + SRC [6] sequence-based 64.11 LSTM [13] sequence-based 78.61 DGCN [9] sequence-based 81.53 WMCNN-LSTM [42] sequence-based 87.10 SAANet [43] sequence-based HoloNet [44] frame-based 44.57…”
BackgroundMethodInfluentialFine-Grained Facial Expression Recognition in the Wild
Liang, L., Lang, C., Li, Y., Feng, S., & Zhao, J. C. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3007327 · Cited by 81 · see citing works
“and CNN+LSTM [32] takes LSTM as temporal network.”
BackgroundFLEPNet: Feature Level Ensemble Parallel Network for Facial Expression Recognition
Karnati, M., Seal, A., Yazidi, A., & Krejcar, O. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3208309 · Cited by 79 · see citing works
“By integrating the properties of facial expressions utilizing DCNN and LSTM, Kim et al. [20] presented a Spatio-temporal feature extraction learning for tackling the FER problem.”
MethodMicro and Macro Facial Expression Recognition Using Advanced Local Motion Patterns
Allaert, B., Bilasco, I. M., & Djéraba, C. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2949559 · Cited by 64 · see citing works
Audio–Visual Fusion for Emotion Recognition in the Valence–Arousal Space Using Joint Cross-Attention
Praveen, R. G., Cardinal, P., & Granger, É. (2023) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2022.3233083 · Cited by 62 · see citing works
“tion with Recurrent Neural Networks (RNN) to capture the effective latent appearance representation, along with temporal dynamics [43].”
MethodEvaluation of the Spatio-Temporal Features and GAN for Micro-Expression Recognition System
Liong, S., Gan, Y. S., Zheng, D., Li, S.-M., Xu, H.-X., Zhang, H., Lyu, R.-K., & Liu, K. (2020) · Journal of Signal Processing Systems · doi:10.1007/s11265-020-01523-4 · Cited by 58 · see citing works
“Due to the emergence of deep learning, the Softmax classifier, normally served as the final fully connected layer, has been employed in the recent works [64, 75, 67, 66].”
BackgroundMethodA Robust Facial Expression Recognition Algorithm Based on Multi-Rate Feature Fusion Scheme
Park, S.-J., Kim, B., & Chilamkurti, N. (2021) · Sensors · doi:10.3390/s21216954 · Cited by 59 · see citing works
“A CNN-RNN or CNN-LSTM [39,40] structure is one of the examples.”
MethodCoReD: Generalizing Fake Media Detection with Continual Representation using Distillation
Kim, M., Tariq, S., & Woo, S. S. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475535 · Cited by 56 · see citing works
Cross Attentional Audio-Visual Fusion for Dimensional Emotion Recognition
Praveen, R. G., Granger, É., & Cardinal, P. (2021) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg52635.2021.9667055 · Cited by 55 · see citing works
Methods for Facial Expression Recognition with Applications in Challenging Situations
Pise, A. A., Alqahtani, M. A., Verma, P., Purushothama, K., Karras, D. A., Prathibha, S., & Halifa, A. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/9261438 · Cited by 64 · see citing works
“[111] provide a better level of precision than the one that was performed by the Kim group [118].”
MethodOn Driver Behavior Recognition for Increased Safety: A Roadmap
Davoli, L., Martalò, M., Cilfone, A., Belli, L., Ferrari, G., Presta, R., Montanari, R., Mengoni, M., Giraldi, L., Amparore, E. G., et al. (2020) · Safety · doi:10.3390/safety6040055 · Cited by 56 · see citing works
“Hybrid frameworks also exist, in which different architectures’ characteristics are combined together in order to achieve better results [39].”
BackgroundSTCAM: Spatial-Temporal and Channel Attention Module for Dynamic Facial Expression Recognition
Chen, W., Zhang, D., Li, M., & Lee, D.-J. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3027340 · Cited by 54 · see citing works
“Some research focused on modeling the temporal correlation among facial expression frames in the video by modeling temporal correlation separately from spatial correlation [12], [13], [14], [15].”
BackgroundMethodA deep-learning-based facial expression recognition method using textural features
Mukhopadhyay, M., Dey, A., & Kahali, S. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-08005-7 · Cited by 55 · see citing works
Pei, J., & Peng, S. (2019) · Traitement du signal · doi:10.18280/ts.360611 · Cited by 54 · see citing works
Dynamic Objectives Learning for Facial Expression Recognition
Wen, G., Chang, T., Li, H., & Jiang, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2966858 · Cited by 53 · see citing works
“A multi-velocity layers are also applied to modify the deep neural network topology for facial expression recognition in video clips [49], while the spatio-temporal feature representation learning is applied to implement expression intensity variations [50].”
Ngo, Q. T., & Yoon, S. (2020) · Sensors · doi:10.3390/s20092639 · Cited by 50 · see citing works
Geometry-Aware Facial Expression Recognition via Attentive Graph Convolutional Networks
Zhao, R., Liu, T., Huang, Z.-X., Lun, D., & Lam, K.-M. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3088895 · Cited by 48 · see citing works
“In recent decades, studies [2], [3], [4], [5], [6], [7], [8], [9] on affective computing, pattern recognition, and human-computer interaction have made great efforts to perform automatic recognition and analysis on human emotions from facial images.”
BackgroundSun, X. N., Xia, P., & Ren, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.127 · Cited by 49 · see citing works
Compression of dynamic tactile information in the human hand
Shao, Y., Vincent Hayward, B., & Visell, Y. (2020) · Science Advances · doi:10.1126/sciadv.aaz1158 · Cited by 48 · see citing works
Spatial-Temporal Graphs Plus Transformers for Geometry-Guided Facial Expression Recognition
Zhao, R., Liu, T., Huang, Z., Lun, D. P.-K., & Lam, K. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3181736 · Cited by 44 · see citing works
“Kim et al. [26] presented an LSTM framework to learn expression-intensity invariant features through facial sequences for FER.”
BackgroundA Convolutional Neural Network for Compound Micro-Expression Recognition
Zhao, Y., & Xu, J. (2019) · Sensors · doi:10.3390/s19245553 · Cited by 39 · see citing works
E2‐capsule neural networks for facial expression recognition using AU‐aware attention
Cao, S., Yao, Y., & An, G. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2020.0063 · Cited by 39 · see citing works
Jin, X., Lai, Z., & Jin, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3101820 · Cited by 39 · see citing works
Mutual information regularized identity-aware facial expression recognition in compressed video
Liu, X., Jin, L., Han, X., & You, J. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108105 · Cited by 36 · see citing works
An Automatic Recognition Method for Students’ Classroom Behaviors Based on Image Processing
Huang, W., Li, N., Qiu, Z., Jiang, N., Wu, B., & Лю, Б. (2020) · Traitement du signal · doi:10.18280/ts.370318 · Cited by 36 · see citing works
“Kim et al. [18] proposed a method that processes human motions in real time: Based on the information of human motions, the regions of interest (ROIs) were determined in specified parts, and the oriented gradient of each ROI was calculated; After that, human motions were classified by a…”
MethodBaddar, W. J., & Ro, Y. M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013215 · Cited by 6 · see citing works
Self-citationResponsible AI: Gender bias assessment in emotion recognitionPreprint
Domnich, A., & Anbarjafari, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11436 · Cited by 65 · see citing works
“[78] is another good example of usage a combination of CNN and LSTM.”
BackgroundZhu, X., & Chen, Z. (2019) · The Visual Computer · doi:10.1007/s00371-019-01660-3 · Cited by 34 · see citing works
“Although the spatiotemporal features add dynamic features compared to the static features, they may be negatively affected by the different characteristic of the facial expression (e.g., different transition duration and/or transition type such as onset or offset) [9].”
BackgroundMethodSG-DSN: A Semantic Graph-based Dual-Stream Network for facial expression recognition
Liu, Y. M., Zhang, X., Zhou, J., & Fu, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.017 · Cited by 34 · see citing works
Facial Expression Recognition in the Wild Using Face Graph and Attention
Kim, H.-J., Lee, J.-H., & Ko, B. C. (2023) · IEEE Access · doi:10.1109/access.2023.3286547 · Cited by 32 · see citing works
“In contrast, Jacob and Stenger [12] extracted facial AUs by using an attentionmapwithout applying landmarks.”
BackgroundMethodInfluentialMoorthy, S., & Moon, Y.-K. (2025) · Mathematics · doi:10.3390/math13071100 · Cited by 31 · see citing works
“Kim et al. [52] proposed a frame-based approach to FER, where spatial image features from representative expression-state frames are learned using a convolutional neural network.”
Reducing Videoconferencing Fatigue through Facial Emotion Recognition
Rößler, J., Sun, J., & Gloor, P. A. (2021) · Future Internet · doi:10.3390/fi13050126 · Cited by 32 · see citing works
Adaptive Thresholding of CNN Features for Maize Leaf Disease Classification and Severity Estimation
Mafukidze, H. D., Owomugisha, G., Otim, D., Nechibvute, A., Nyamhere, C., & Mazunga, F. (2022) · Applied Sciences · doi:10.3390/app12178412 · Cited by 31 · see citing works
A Graph-Structured Representation with BRNN for Static-based Facial Expression Recognition
Zhong, L., Bai, C., Li, J., Chen, T., Li, S., & Liu, Y. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756615 · Cited by 29 · see citing works
Deep Neural Network Architecture: Application for Facial Expression Recognition
Villanueva, M. G., & Zavala, S. R. (2020) · IEEE Latin America Transactions · doi:10.1109/tla.2020.9099774 · Cited by 29 · see citing works
“Algunos de los estudios representativos que utilizan una combinaci´on de CNN y LSTM (RNN) son: [18], [19], [20], [14], [15], [16], [21], [22].”
BackgroundLi, Q.-L., Zhang, C., Shangguan, W., Wei, Z.-W., Yuan, H., Zhu, J.-L., Li, X.-N., Li, L., Li, G., Liu, P., et al. (2023) · Expert systems with applications · doi:10.1016/j.eswa.2023.122917 · Cited by 24 · see citing works
Tang, J., Li, L., Tang, M., & Xie, J. (2022) · Signal Image and Video Processing · doi:10.1007/s11760-022-02286-0 · Cited by 27 · see citing works
“[13] proposed a new temporal feature learning method.”
BackgroundLARNet: Real-Time Detection of Facial Micro Expression Using Lossless Attention Residual Network
Hashmi, M. F., Ashish, B. K. K., Sharma, V., Keskar, A. G., Bokde, N. D., Yoon, J. H., & Geem, Z. W. (2021) · Sensors · doi:10.3390/s21041098 · Cited by 27 · see citing works
“[34] proposed a model consisting of CNN and long short-term memory (LSTM) to manage spatial and temporal information.”
Verma, M., Lubal, P., Vipparthi, S. K., & Abdel-Mottaleb, M. S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00475 · Cited by 26 · see citing works
“In the literature, both traditional [18, 6, 37, 17, 5] and CNN-based algorithms [8, 11, 13, 32, 9, 29, 28, 27] have been effective for micro-expression recognition (MER).”
BackgroundNon-Linearities Improve OrigiNet based on Active Imaging for Micro Expression Recognition
Verma, M., Vipparthi, S. K., & Singh, G. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207718 · Cited by 26 · see citing works
“The Spontaneous Actions and Micro-Movements (SAMM) [35] is a newer database recorded at 200 fps and resolution set to 2,040×1,088 pixels.”
BackgroundMethodInfluentialEmotion Recognition Using EEG Signals and Audiovisual Features with Contrastive Learning
Lee, J.-H., Kim, J. Y., & Kim, H. (2024) · Bioengineering · doi:10.3390/bioengineering11100997 · Cited by 25 · see citing works
Li, Z., Zhang, T., Jing, X., & Wang, Y. (2020) · Alexandria Engineering Journal · doi:10.1016/j.aej.2020.10.061 · Cited by 25 · see citing works
Zhao, L., Ge, Y., Guo, S., Li, H., Li, X., Sun, L., & Chen, J. (2024) · Forest Ecology and Management · doi:10.1016/j.foreco.2024.121771 · Cited by 24 · see citing works
Identity-aware Facial Expression Recognition in Compressed Video
Liu, X., Jin, L., Han, X., Lu, J., You, J., & Kong, L. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412820 · Cited by 24 · see citing works
“The recursive neural network (RNN), 3D convolutional, and non-local network can be used to derive information from sequences [3], [4], [5].”
BackgroundMethodInfluentialGajjala, V. R., Reddy, S. P. T., Mukherjee, S., & Dubey, S. R. (2021) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3490035.3490260 · Cited by 24 · see citing works
Efficient neural architecture search for emotion recognition
Verma, M., Mandal, M., Reddy, S. K., Meedimale, Y. R., & Vipparthi, S. K. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119957 · Cited by 23 · see citing works
“Particularly, proposed EmoNAS is 155× and 20× less than the ViT and Squeeze viT Kim et al. (2022) FER models, respectively.”
MethodAttended Relation Feature Representation of Facial Dynamics for Facial Authentication
Kim, S. T., & Ro, Y. M. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2885276 · Cited by 23 · see citing works
“Recently, several research efforts have been made on facial dynamic encoding for face analysis [18]–[27].”
BackgroundSelf-citationYi, J., Chen, A., Cai, Z., Sima, Y., Zhou, M., & Wu, X. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105540 · Cited by 23 · see citing works
AutoMER: Spatiotemporal Neural Architecture Search for Microexpression Recognition
Verma, M., Reddy, M. S. K., Meedimale, Y. R., Mandal, M., & Vipparthi, S. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3072290 · Cited by 22 · see citing works
“Recently, the deep learning algorithms based on CNN [1], [2], [16]–[19] have also achieved impressive results in MER.”
BackgroundMethodDeep Learning-Based Holistic Speaker Independent Visual Speech Recognition
Nemani, P., Krishna, G. S., Nikhil, R., Sai, B. D. S., & Kumar, S. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3220190 · Cited by 22 · see citing works
“D Spatio-Temporal Feature Representation: As discussed in Section I, there is a need for the fine-scale representation of the spatio-temporal features of the different video frames [32], [33], [34].”
BackgroundA micro-expression recognition algorithm based on feature enhancement and attention mechanisms
Tang, M., Ling, M., Tang, J., & Hu, J. (2023) · Virtual Reality · doi:10.1007/s10055-023-00808-w · Cited by 22 · see citing works
“The network proposed by Kim et al. (2019) combines CNN and Long Short-Term Memory ( LSTM ).”
MethodEmotion Identification in Movies through Facial Expression Recognition
Almeida, J. E., Vilaça, L., Teixeira, I. N., & Viana, P. (2021) · Applied Sciences · doi:10.3390/app11156827 · Cited by 21 · see citing works
Yang, X., Yan, H., Zhang, A., Xu, P., Pan, S. H., Vai, M. I., & Gao, Y. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105921 · Cited by 21 · see citing works
Zhang, J., Sun, G., Zheng, K., Mazhar, S., Fu, X., Li, Y., & Yu, H. L. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3163211 · Cited by 20 · see citing works
Human Emotion Recognition in Video Using Subtraction Pre-Processing
He, Z., Tian, J. W., Basu, A., Soraghan, J. J., Caterina, G. D., & Petropoulakis, L. (2019) · International Conference on Machine Learning and Computing (ICMLC) · doi:10.1145/3318299.3318321 · Cited by 20 · see citing works
On-the-Fly Facial Expression Prediction Using LSTM Encoded Appearance-Suppressed Dynamics
Baddar, W. J., Lee, S., & Ro, Y. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2957465 · Cited by 19 · see citing works
Self-citationMultimodal emotion recognition using SDA-LDA algorithm in video clips
Tiwari, P., Rathod, H., Thakkar, S., & Darji, A. D. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03529-7 · Cited by 20 · see citing works
“Based on SAVEE database, Kim et al. (2017) proposed an Informed Segmentation and Labelling Approach (ISLA) that uses speech signals to alter the dynamics of the lower and upper face regions.”
MethodDeep Temporal–Spatial Aggregation for Video-Based Facial Expression Recognition
Pan, X., Guo, W., Guo, X., Li, W., Xu, J., & Wu, J. (2019) · Symmetry · doi:10.3390/sym11010052 · Cited by 19 · see citing works
“The recent deep spatial–temporal networks are designed to recognize facial expression in video and improve recognition performance [39], extracting deep spatial features and temporal features from the video.”
BackgroundHu, M. Q., Ge, P., Wang, X., Lin, H. Z., & Ren, F. (2021) · The Visual Computer · doi:10.1007/s00371-021-02136-z · Cited by 19 · see citing works
“Therefore, some researchers use recurrent neural network (RNN), long short-term memory network (LSTM) or their improved versions [40] to capture temporal information because they perform well in processing sequences with contextual information [41–46].”
BackgroundVAE-Based Adversarial Multimodal Domain Transfer for Video-Level Sentiment Analysis
Wang, Y., Wu, J., Furumai, K., Wada, S., & Kurihara, S. (2022) · IEEE Access · doi:10.1109/access.2022.3174215 · Cited by 19 · see citing works
Dynamic Facial Expression Feature Learning Based on Sparse RNN
Zhi, R., & Wan, M. (2019) · IEEE Joint International Information Technology and Artificial Intelligence Conference (ITAIC) · doi:10.1109/itaic.2019.8785844 · Cited by 18 · see citing works
“Similarly, the characteristics of facial expressions were encoded in two parts in [8].”
BackgroundVariational restricted Boltzmann machines to automated anomaly detection
Demertzis, K., Iliadis, L., Pimenidis, E., & Kikiras, P. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07060-4 · Cited by 18 · see citing works
“Therefore, the solutions are related to the maximization of the Evidence Lower Bound (ELBO), which is equivalent to minimizing the KL deviation, so that [40, 63]:”
MethodViTCN: Hybrid Vision Transformer with Temporal Convolution for Multi-Emotion Recognition
Zakieldin, K., Khattab, R., Ibrahim, E., Arafat, E., Ahmed, N. K., & Hemayed, E. E. (2024) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-024-00436-5 · Cited by 18 · see citing works
Bohi, A., Boudouri, Y. E., & Sfeir, I. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10938-0 · Cited by 18 · see citing works
A multi-scale multi-attention network for dynamic facial expression recognition
Xia, X., Yang, L., Wei, X.-Y., Hichem Sahli, B., & Jiang, D. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00849-8 · Cited by 18 · see citing works
“One common approach is the cascaded framework, where a CNN model is first trained for deep spatial feature learning, followed by a LSTM model for temporal sequence modelling [18].”
MethodRETRACTED ARTICLE: Recognition of emotions in speech using deep CNN and RESNET
Lakshmi, K. L., Muthulakshmi, P., Nithya, A. A., Jeyavathana, R. B., Usharani, R., Das, N. S., & Devi, G. N. R. (2023) · Soft Computing · doi:10.1007/s00500-023-07969-5 · Cited by 18 · see citing works
RetractedDeep 3D Convolutional Neural Network for Facial Micro-Expression Analysis from Video Images
Talluri, K. K., Fiedler, M.-A., & Al-Hamadi, A. K. (2022) · Applied Sciences · doi:10.3390/app122111078 · Cited by 17 · see citing works
“Usually, the spatial information was extracted with CNNs, and subsequently, the features obtained were given into an LSTM to analyze the temporal information [13,14].”
MethodOrtmann, T., Wang, Q., & Putzar, L. (2024) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii63134.2024.00014 · Cited by 16 · see citing works
“While more works focus on static FER, processing single images, dynamic FER systems have demonstrated that including temporal features using multiple frames can be beneficial [11].”
Song Playlist Generator System Based on Facial Expression and Song Mood
Patel, K., & Gupta, R. K. (2021) · International Conference on Artificial Intelligence and Machine Vision (AIMV) · doi:10.1109/aimv53313.2021.9670976 · Cited by 16 · see citing works
FERFM: An Enhanced Facial Emotion Recognition System Using Fine-tuned MobileNetV2 Architecture
Kaur, S., & Kulkarni, N. (2023) · IETE Journal of Research · doi:10.1080/03772063.2023.2202158 · Cited by 15 · see citing works
“[24] CNN spatial features followed by LSTM temporal features”
Human Emotion Recognition Based on Spatio-Temporal Facial Features Using HOG-HOF and VGG-LSTM
Chouhayebi, H., Mahraz, M. A., Riffi, J., Tairi, H., & Alioua, N. (2024) · Computers · doi:10.3390/computers13040101 · Cited by 15 · see citing works
Open-Set Video-based Facial Expression Recognition with Human Expression-sensitive Prompting
Liu, Y., Huang, Y., Liu, S., Zhan, Y., Chen, Z., & Chen, Z. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681583 · Cited by 15 · see citing works
“Currently, many studies use structures like LSTM [19, 22], 3D-CNN [24, 26, 42], Transformer [25, 43] to extract spatio-temporal expression information for V-FER.”
MethodAn Adaptive Optimization Algorithm Based on Hybrid Power and Multidimensional Update Strategy
Hu, J., & Zheng, W. (2019) · IEEE Access · doi:10.1109/access.2019.2897639 · Cited by 15 · see citing works
“A new learning method of spatiotemporal correlation characteristic is proposed to effectively train the model by encoding the spatio-temporal correlation information of the sequence data [20], [21].”
MethodZhen, P., Chen, H.-B., Cheng, Y., Ji, Z., Liu, B., & Yu, H. (2021) · ACM Trans. Internet Things · doi:10.1145/3464941 · Cited by 16 · see citing works
“Consequently, LSTM is widely adopted for FER with temporal information [13, 15, 27].”
BackgroundMethodInfluentialEmotion, Age and Gender Recognition using SURF, BRISK, M-SVM and Modified CNN
Sikder, J., Datta, N., Tripura, S., & Das, U. K. (2022) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet55527.2022.9872771 · Cited by 14 · see citing works
Huang, Z., Zhu, Y. M., Li, H., & Yang, D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108535 · Cited by 15 · see citing works
- Authors
- Kim, D. H., Baddar, W. J., Jang, J., & Ro, Y. M.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2019-04-01
- Volume
- 10
- Issue
- 2
- Pages
- 223-236
- ISSN
- 1949-3045 (ISSN-L)
- 2371-9850
- OpenAlex
- W2606933083
- Cited by
- 257
- References
- 61
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 33 · 2023: 36 · 2022: 56 · 2021: 39 · 2020: 28 · 2019: 26 · 2018: 13 · 2017: 3
- FWCI
- 24.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Affective Computing (publishedVersion)
- KAIST Institutional Repository (KAIST) (submittedVersion)
- Funders
- National Research Foundation of Korea
- Korea government (2015R1A2A2A01005724)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- facial expression recognition, multi-objective learning, long short-term memory, micro-expressions, class separability, affective computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review